Multi-frequency phase shift measurement method, device, storage medium and equipment

Abstract

The embodiments of the present application disclose a multi-frequency phase shift measurement method, device, storage medium and equipment, which belong to the field of optical detection technology. The method includes: controlling a projector to project a target object using a plurality of sinusoidal phase shift stripes of different frequencies; determining the optimal exposure time for a camera to shoot a plurality of sinusoidal phase shift stripes of different frequencies according to the projection intensity ratio of the sinusoidal phase shift stripes of different frequencies on a reference plane; obtaining a sequence of images of a plurality of sinusoidal phase shift stripes of different frequencies taken by the camera at the optimal exposure time of the sinusoidal phase shift stripes of different frequencies; calculating the phase diagram of the sinusoidal phase shift stripes of different frequencies according to the sequence of images of the sinusoidal phase shift stripes of different frequencies; determining the three-dimensional information of the target object according to the phase diagram of the sinusoidal phase shift stripes of different frequencies. The embodiments of the present application can only use the optimal exposure time for exposure, saving computing resources.

Description

Technical Field

[0001] The present application relates to the field of optical detection, and in particular to a multi-frequency phase shift measurement method, device, storage medium and equipment. Background Art

[0002] 3D Automatic Optic Inspection (3D AOI) systems generally use the principle of multi-frequency phase-shift imaging to obtain high-precision 3D point cloud data of the target area, and are often used for high-speed, high-precision quantitative defect detection on printed circuit board (PCB) circuit board production lines.

[0003] In the related art, a projector can project a sinusoidal phase-shifted fringe sequence of different frequencies onto a target object as coding information to mark the target object, and then a camera can capture the sinusoidal phase-shifted fringe sequence. Since the sinusoidal fringe sequence is modulated by the height information of the target object and deformed, the camera captures the deformed sinusoidal fringe sequence and performs phase decoding to obtain multiple wrapped phase images. The absolute phase image is obtained by unwrapping multiple wrapped phase images, and the three-dimensional information of the target object can be calculated using the absolute phase and the posture relationship between the camera and the projector.

[0004] In order to improve the dynamic range of the 3D AOI system and improve the imaging capability of the 3D AOI system in high reflectivity areas, it is usually necessary to expose the sinusoidal phase shift stripes of each frequency multiple times, select the intensity sequence corresponding to the best exposure mode for phase calculation, and then calculate the absolute phase map based on the multi-frequency phase map. However, multiple exposures of the sinusoidal phase shift stripes of each frequency will lead to cumbersome operation, increase the difficulty of multi-frequency phase shift measurement, and the increased data volume of multiple exposures will also occupy more computing resources. Summary of the invention

[0005] The embodiments of the present application provide a multi-frequency phase shift measurement method, device, storage medium and equipment, which can solve the problem that the amount of data increased by multiple exposures of sinusoidal phase shift stripes at each frequency occupies a lot of computing resources. The technical solution is as follows:

[0006] In a first aspect, an embodiment of the present application provides a multi-frequency phase shift measurement method, comprising:

[0007] Controlling the projector to project a plurality of sinusoidal phase-shifted fringes of different frequencies onto the target object;

[0008] Determining respectively the optimal exposure time for a camera to shoot the multiple sinusoidal phase-shifted fringes of different frequencies according to the projection intensity ratios of the multiple sinusoidal phase-shifted fringes of different frequencies on the reference plane;

[0009] Acquire a sequence of images of the plurality of sinusoidal phase-shifted fringes with different frequencies respectively captured by the camera using the optimal exposure time of the plurality of sinusoidal phase-shifted fringes with different frequencies;

[0010] Calculating phase diagrams of the plurality of sinusoidal phase-shifted fringes of different frequencies respectively according to the image sequences of the plurality of sinusoidal phase-shifted fringes of different frequencies;

[0011] The three-dimensional information of the target object is determined according to the phase diagram of the multiple sinusoidal phase-shifted fringes of different frequencies.

[0012] Optionally, determining the optimal exposure time for a camera to shoot the plurality of sinusoidal phase-shifted fringes with different frequencies respectively according to the projection intensity ratio of the plurality of sinusoidal phase-shifted fringes with different frequencies on a reference plane includes:

[0013] Determining an optimal exposure time for the camera to shoot the sinusoidal phase-shifted fringes of the first frequency according to an exposure area in a shooting picture of the sinusoidal phase-shifted fringes of the first frequency by the camera;

[0014] The optimal exposure time of the sinusoidal phase-shifted fringes except the first frequency is determined according to the optimal exposure time of the sinusoidal phase-shifted fringes of the first frequency and the projection intensity ratio of the multiple sinusoidal phase-shifted fringes of different frequencies on the reference plane.

[0015] Optionally, the first frequency is the highest frequency among the multiple different frequencies.

[0016] Optionally, determining the optimal exposure time for the camera to shoot the sinusoidal phase-shifted fringes of the first frequency according to the exposure area in the shooting picture of the sinusoidal phase-shifted fringes of the first frequency by the camera includes:

[0017] Determine an exposure area in a picture captured by the camera using a plurality of different exposure times for the sinusoidal phase-shifted fringes of the first frequency;

[0018] The exposure time corresponding to the most exposed area in the shooting picture of the sinusoidal phase-shifted stripes of the first frequency is used as the optimal exposure time of the sinusoidal phase-shifted stripes of the first frequency.

[0019] Optionally, the calculating, according to the image sequences of the plurality of sinusoidal phase-shifted fringes with different frequencies, the phase diagrams of the plurality of sinusoidal phase-shifted fringes with different frequencies respectively includes:

[0020] Determine the number of pixels at a plurality of pixel positions in a sequence of images of the plurality of sinusoidal phase-shifted stripes of different frequencies;

[0021] Determine pixel labels corresponding to the plurality of pixel positions according to the number of pixels of the plurality of pixel positions in the image sequence of the plurality of sinusoidal phase-shifted fringes of different frequencies;

[0022] Phase diagrams of the multiple sinusoidal phase-shifted fringes of different frequencies are calculated respectively according to the pixel marks corresponding to the multiple pixel positions.

[0023] Optionally, determining the pixel labels corresponding to the multiple pixel positions according to the number of pixels at the multiple pixel positions in the image sequence of the multiple sinusoidal phase-shifted fringes with different frequencies includes:

[0024] Compare the number of pixels of the plurality of pixel positions in the image sequence of the plurality of sinusoidal phase-shifted fringes of different frequencies with a threshold value respectively, and determine a comparison result;

[0025] According to the comparison result, pixel labels corresponding to the plurality of pixel positions are determined.

[0026] Optionally, determining the three-dimensional information of the target object according to the phase diagram of the multiple sinusoidal phase-shifted fringes of different frequencies includes:

[0027] The phase diagram of the sinusoidal phase shift fringes other than the first frequency is used to unwrap the phase diagram of the sinusoidal phase shift fringes of the first frequency to determine the three-dimensional information of the target object.

[0028] In a second aspect, an embodiment of the present application provides a multi-frequency phase shift measurement method and device, including:

[0029] A projection module, used to control a projector to project a target object using a plurality of sinusoidal phase-shifted fringes of different frequencies;

[0030] An adjustment module, used to determine the optimal exposure time for a camera to shoot the multiple sinusoidal phase-shifted fringes with different frequencies, respectively, according to the projection intensity ratio of the multiple sinusoidal phase-shifted fringes with different frequencies on the reference plane;

[0031] An acquisition module, used for acquiring a sequence of images of the plurality of sinusoidal phase-shifted fringes with different frequencies respectively captured by the camera using the optimal exposure time of the plurality of sinusoidal phase-shifted fringes with different frequencies;

[0032] A calculation module is used to calculate the phase diagrams of the multiple sinusoidal phase-shifted fringes with different frequencies respectively according to the image sequences of the multiple sinusoidal phase-shifted fringes with different frequencies; and determine the three-dimensional information of the target object according to the phase diagrams of the multiple sinusoidal phase-shifted fringes with different frequencies.

[0033] Optionally, the adjustment module is specifically used to determine the optimal exposure time of the camera for shooting the sinusoidal phase-shifted stripes of the first frequency based on the exposure area in the shooting picture of the sinusoidal phase-shifted stripes of the first frequency by the camera; and determine the optimal exposure time of the sinusoidal phase-shifted stripes other than the first frequency based on the optimal exposure time of the sinusoidal phase-shifted stripes of the first frequency and the projection intensity ratio of the multiple sinusoidal phase-shifted stripes of different frequencies on the reference plane.

[0034] Optionally, the first frequency is the highest frequency among the multiple different frequencies.

[0035] Optionally, the adjustment module is specifically used to determine the exposure area in the picture captured by the camera using multiple different exposure times for the sinusoidal phase-shift stripes of the first frequency; and use the exposure time corresponding to the most exposed area in the picture captured by the sinusoidal phase-shift stripes of the first frequency as the optimal exposure time for the sinusoidal phase-shift stripes of the first frequency.

[0036] Optionally, the calculation module is specifically used to determine the number of pixels at multiple pixel positions in the image sequence of the multiple sinusoidal phase-shifted stripes with different frequencies; determine the pixel marks corresponding to the multiple pixel positions according to the number of pixels at the multiple pixel positions in the image sequence of the multiple sinusoidal phase-shifted stripes with different frequencies; and calculate the phase diagrams of the multiple sinusoidal phase-shifted stripes with different frequencies respectively according to the pixel marks corresponding to the multiple pixel positions.

[0037] Optionally, the calculation module is specifically used to compare the number of pixels of the multiple pixel positions in the image sequence of the multiple sinusoidal phase-shifted fringes of different frequencies with a threshold value, respectively, to determine a comparison result;

[0038] According to the comparison result, pixel labels corresponding to the plurality of pixel positions are determined.

[0039] Optionally, the calculation module is specifically configured to unpack the phase diagram of the sinusoidal phase shift fringes of the first frequency using the phase diagram of the sinusoidal phase shift fringes other than the first frequency to determine the three-dimensional information of the target object.

[0040] In a third aspect, an embodiment of the present application provides a computer program product, including computer instructions, which, when executed by a processor, implement any method described in the first aspect.

[0041] In a fourth aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0042] In a fifth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0043] In the embodiment of the present application, a projector is controlled to project a target object using a plurality of sinusoidal phase shift stripes of different frequencies; the optimal exposure time for the camera to shoot the plurality of sinusoidal phase shift stripes of different frequencies is determined according to the projection intensity ratio of the plurality of sinusoidal phase shift stripes of different frequencies on the reference plane; a sequence of pictures of the plurality of sinusoidal phase shift stripes of different frequencies shot by the camera using the optimal exposure time of the plurality of sinusoidal phase shift stripes of different frequencies is obtained; the phase diagrams of the plurality of sinusoidal phase shift stripes of different frequencies are calculated according to the sequence of pictures of the plurality of sinusoidal phase shift stripes of different frequencies; and the three-dimensional information of the target object is determined according to the phase diagrams of the plurality of sinusoidal phase shift stripes of different frequencies. Since the optimal exposure time is determined, the camera can directly use the optimal exposure time corresponding to the sinusoidal phase shift stripes of different frequencies to shoot the sequence of pictures of the sinusoidal phase shift stripes of different frequencies respectively, thereby avoiding the increase in the amount of data caused by multiple exposures of the sinusoidal phase shift stripes of each frequency, and saving computing resources. At the same time, only exposing the sinusoidal phase shift stripes of each frequency only once also simplifies the operation steps and reduces the difficulty of multi-frequency phase shift measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A schematic diagram of a multi-frequency phase shift measurement provided in an embodiment of the present application;

[0046] Figure 2 A schematic diagram of an existing high dynamic range three-dimensional measurement method;

[0047] Figure 3 A schematic diagram of a flow chart of a multi-frequency phase shift measurement method provided in an embodiment of the present application;

[0048] Figure 4 A schematic flow chart of another multi-frequency phase shift measurement method provided in an embodiment of the present application;

[0049] Figure 5 A schematic flow chart of another multi-frequency phase shift measurement method provided in an embodiment of the present application;

[0050] Figure 6 A flowchart of another multi-frequency phase shift measurement method provided in an embodiment of the present application;

[0051] Figure 7 A schematic diagram of the structure of a multi-frequency phase shift measurement device provided in an embodiment of the present application;

[0052] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the objectives, technical solutions and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0054] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0055] When the following description refers to the drawings, unless otherwise indicated, the same numbers in multiple different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0056] In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0057] Embodiment 1 of the present application provides a multi-frequency phase shift measurement method.

[0058] Embodiment 2 of the present application provides a method for determining the optimal exposure time for a camera to capture multiple sinusoidal phase-shifted stripes of different frequencies.

[0059] Embodiment 3 of the present application provides a method for calculating the phase diagram of sinusoidal phase-shifted fringes of multiple different frequencies.

[0060] Embodiment 4 of the present application provides a method for determining three-dimensional information of a target object.

[0061] It should be noted that due to space limitations, this application description does not list all optional implementation methods. After reading this application description, those skilled in the art should be able to understand that as long as the technical features do not contradict each other, any combination of technical features can constitute an optional implementation method.

[0062] Non-contradictory technical features recorded in a plurality of different embodiments may also be arbitrarily combined to form optional implementation modes.

[0063] For example, Example 4 records: a method for determining three-dimensional information of a target object. In order to control the length of the specification of this application, this feature is not recorded in Example 1, Example 2, and Example 3. However, after reading the specification of this application, a person skilled in the art should be able to think that the multi-frequency phase shift measurement method provided in Example 2 and Example 3 may also include this feature.

[0064] The multi-frequency phase shift measurement technology is first described below.

[0065] Figure 1 A schematic diagram of a multi-frequency phase shift measurement provided in an embodiment of the present application, such as Figure 1 As shown, the terminal device 101 can control the projector 102 to project a sinusoidal phase-shifted fringe sequence of different frequencies onto the target object as the encoding information to mark the target object. Subsequently, the terminal device 101 controls the camera 103 to shoot the sinusoidal phase-shifted fringe sequence and obtains the sinusoidal phase-shifted fringe sequence captured by the camera 103. Since the sinusoidal fringe sequence is modulated by the height information of the target object and deformed, after the terminal device 101 obtains the deformed sinusoidal fringe sequence captured by the camera 103, it can perform phase decoding on the deformed sinusoidal fringe sequence to obtain multiple wrapped phase images. Finally, the terminal device 101 performs an unwrapping operation on the multiple wrapped phase images to obtain an absolute phase image, and the three-dimensional information of the target object can be calculated using the absolute phase and the posture relationship between the camera and the projector.

[0066] In order to improve the dynamic range of the 3D AOI system and improve the imaging capability of the 3D AOI system in high reflectivity areas, it is usually necessary to perform multiple exposures of the sinusoidal phase shift fringes of each frequency, select the intensity sequence corresponding to the best exposure mode for phase calculation, and then calculate the absolute phase map based on the multi-frequency phase map. For example, Figure 2 Schematic diagram of existing high dynamic range three-dimensional measurement methods, such as Figure 2 As shown, the intensity sequence corresponding to the optimal exposure mode is the maximum intensity sequence that does not exceed the upper limit of the sensor dynamic range. For a commonly used 8-bit camera, the upper limit of its dynamic range is 255.

[0067] However, multiple exposures of the sinusoidal phase-shifted fringes at each frequency result in a cumbersome operation process, which increases the difficulty of multi-frequency phase-shift measurement. The increased amount of data from multiple exposures also takes up more computing resources.

[0068] To solve the above problems, the embodiments of the present application provide a multi-frequency phase shift measurement method, device, storage medium and equipment. In the present application, according to the projection intensity ratio of multiple sinusoidal phase shift fringes of different frequencies on the reference plane, the optimal exposure time for the camera to shoot multiple sinusoidal phase shift fringes of different frequencies is determined respectively, so that the camera can directly use the optimal exposure time corresponding to the sinusoidal phase shift fringes of different frequencies to shoot the image sequence of sinusoidal phase shift fringes of different frequencies respectively, avoiding the increase of data volume caused by multiple exposures of sinusoidal phase shift fringes of each frequency, and saving computing resources.

[0069] It can be understood that the above-mentioned multi-frequency phase shift measurement method can be implemented by the multi-frequency phase shift measurement device provided in the embodiment of the present application. The multi-frequency phase shift measurement device can be, for example, the above-mentioned terminal device. The following takes a terminal device integrated or installed with relevant execution code as an example to describe the technical solution of the embodiment of the present application in detail with a specific embodiment. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0070] The following will be combined with the attached Figure 3 -Attached Figure 6 , the multi-frequency phase shift measurement method provided in the embodiment of the present application is introduced in detail.

[0071] Example 1

[0072] The embodiment of the present application discloses a multi-frequency phase shift measurement method, which is applied to a terminal device.

[0073] Figure 3 A flow chart of a multi-frequency phase shift measurement method provided in an embodiment of the present application is shown below in conjunction with the attached Figure 3 , the multi-frequency phase shift measurement method provided in the embodiment of the present application is introduced in detail. Figure 3 , the method comprises the following steps:

[0074] S201, controlling a projector to project a target object using a plurality of sinusoidal phase-shifted fringes of different frequencies.

[0075] In the present application, when multi-frequency phase shift measurement is required for a target object, the terminal device can send control information to the projector to control the projector to project the target object using multiple sinusoidal phase shift fringes of different frequencies.

[0076] Among them, the sinusoidal phase-shifted fringes can be understood as projection fringes that are in a sinusoidal state and have a phase shift in time.

[0077] It should be understood that the embodiments of the present application do not limit the number and frequency of the sinusoidal phase shift fringes projected by the projector. For example, taking the commonly used three-frequency phase shift method as an example, the projector can project sinusoidal phase shift fringes of three frequencies, namely high-frequency sinusoidal phase shift fringes, medium-frequency sinusoidal phase shift fringes, and low-frequency sinusoidal phase shift fringes. The sampling exponential multiple of the sinusoidal phase shift fringes of different frequencies is a power of ten, that is, the frequency of the high-frequency sinusoidal phase shift fringes is The frequency of the intermediate frequency sinusoidal phase shift fringes and the frequency of the low-frequency sinusoidal phase-shifted fringes The ratio between them is 100:10:1.

[0078] S202, determining optimal exposure times for a camera to shoot multiple sinusoidal phase-shifted fringes with different frequencies according to projection intensity ratios of the multiple sinusoidal phase-shifted fringes with different frequencies on a reference plane.

[0079] In this step, after the terminal device controls the projector to project the target object using multiple sinusoidal phase-shifted stripes of different frequencies, the optimal exposure time for the camera to shoot the multiple sinusoidal phase-shifted stripes of different frequencies can be determined according to the projection intensity ratio of the multiple sinusoidal phase-shifted stripes of different frequencies on the reference plane.

[0080] In some embodiments, the projection intensity ratio of multiple sinusoidal phase shift fringes of different frequencies on a reference plane can be calibrated by the projection frequency-intensity relationship. Exemplarily, after the terminal device controls the projector to project the target object, the average intensity values ​​corresponding to the multiple sinusoidal phase shift fringes of different frequencies can be counted on the reference plane, and then the projection intensity ratio of the multiple sinusoidal phase shift fringes of different frequencies on the reference plane can be calculated. Exemplarily, if the multiple sinusoidal phase shift fringes of different frequencies include the frequency of the high-frequency sinusoidal phase shift fringes, The frequency of the intermediate frequency sinusoidal phase shift fringes and the frequency of the low-frequency sinusoidal phase-shifted fringes Then the projection intensity ratio can be P H :P M :P L .

[0081] It should be understood that the embodiments of the present application do not limit how to determine the optimal exposure time for a camera to shoot multiple sinusoidal phase-shifted stripes with different frequencies. In some embodiments, the optimal exposure time of the sinusoidal phase-shifted stripes of a first frequency among multiple different frequencies can be first determined, and then the optimal exposure time of the sinusoidal phase-shifted stripes other than the first frequency can be determined based on the optimal exposure time of the sinusoidal phase-shifted stripes of the first frequency and the projection intensity ratio of the sinusoidal phase-shifted stripes of multiple different frequencies on the reference plane. The optimal exposure time of the sinusoidal phase-shifted stripes of the first frequency can be determined by the exposure area in the shooting picture of the sinusoidal phase-shifted stripes of the first frequency by the camera.

[0082] Preferably, since high-frequency sinusoidal phase-shifted fringe projection can capture more pixels, in order to more accurately calculate the three-dimensional information of the target object, the sinusoidal phase-shifted fringe of the first frequency can be the highest frequency among multiple different frequencies.

[0083] S203, obtaining a sequence of images of a plurality of sinusoidal phase-shifted fringes of different frequencies respectively captured by a camera using optimal exposure times of the sinusoidal phase-shifted fringes of different frequencies.

[0084] In this step, after the terminal device determines the optimal exposure time for the camera to shoot multiple sinusoidal phase-shifted stripes with different frequencies, the optimal exposure time for the camera to shoot multiple sinusoidal phase-shifted stripes with different frequencies can be sent to the camera, so that the camera can shoot a sequence of pictures of multiple sinusoidal phase-shifted stripes with different frequencies according to the optimal exposure time. Subsequently, when the camera completes shooting, the sequence of pictures of multiple sinusoidal phase-shifted stripes with different frequencies can be sent to the terminal device.

[0085] It should be understood that the image sequence may be a series of images taken for sinusoidal phase shift stripes of the same frequency. The embodiment of the present application does not limit the number of images in the image sequence. In actual situations, the image sequence usually contains no less than three images.

[0086] S204, respectively calculating phase diagrams of a plurality of sinusoidal phase-shifted fringes of different frequencies according to a sequence of images of a plurality of sinusoidal phase-shifted fringes of different frequencies.

[0087] In this step, after the terminal device receives a sequence of images of sinusoidal phase-shifted fringes of multiple different frequencies taken by a camera, the phase diagrams of the sinusoidal phase-shifted fringes of multiple different frequencies can be respectively calculated according to the sequence of images of the sinusoidal phase-shifted fringes of multiple different frequencies.

[0088] It should be understood that the phase map involved in the embodiment of the present application can also be called a dot matrix map, which is an image represented by pixels.

[0089] The embodiments of the present application do not limit how to calculate the phase diagram of multiple sinusoidal phase-shifted fringes of different frequencies. In some embodiments, the terminal device may first determine the number of pixels at multiple pixel positions in a picture sequence of multiple sinusoidal phase-shifted fringes of different frequencies. Subsequently, the terminal device may determine the pixel tags corresponding to the multiple pixel positions based on the number of pixels at the multiple pixel positions in the picture sequence of multiple sinusoidal phase-shifted fringes of different frequencies. Finally, the terminal device may respectively calculate the phase diagram of multiple sinusoidal phase-shifted fringes of different frequencies based on the pixel tags corresponding to the multiple pixel positions.

[0090] S205, determining three-dimensional information of the target object according to a phase diagram of a plurality of sinusoidal phase-shifted fringes of different frequencies.

[0091] In this step, after the terminal device calculates the phase diagrams of the sinusoidal phase shift fringes of multiple frequencies, the three-dimensional information of the target object can be determined according to the phase diagrams of the sinusoidal phase shift fringes of multiple frequencies.

[0092] In some embodiments, the terminal device uses the phase diagram of the sinusoidal phase shift fringes other than the first frequency to unpack the phase diagram of the sinusoidal phase shift fringes of the first frequency to determine the three-dimensional information of the target object.

[0093] It should be understood that by unwrapping the phase map of the sinusoidal phase shift fringes of a certain frequency, an absolute phase map can be obtained. Subsequently, the depth information of the target object can be calculated using the absolute phase map and the posture relationship between the camera and the projector. Finally, the depth information of the target object can be combined with the X-axis and Y-axis coordinates of the pixels to obtain the three-dimensional information of the target object.

[0094] In the multi-frequency phase shift measurement method provided in the embodiment of the present application, a projector is controlled to project a target object using a plurality of sinusoidal phase shift stripes of different frequencies; the optimal exposure time for the camera to shoot a plurality of sinusoidal phase shift stripes of different frequencies is determined according to the projection intensity ratio of the sinusoidal phase shift stripes of different frequencies on the reference plane; a sequence of pictures of the sinusoidal phase shift stripes of different frequencies shot by the camera using the optimal exposure time of the sinusoidal phase shift stripes of different frequencies is obtained; the phase diagrams of the sinusoidal phase shift stripes of different frequencies are calculated according to the sequence of pictures of the sinusoidal phase shift stripes of different frequencies; the three-dimensional information of the target object is determined according to the phase diagrams of the sinusoidal phase shift stripes of different frequencies. Since the optimal exposure time is determined, the camera can directly use the optimal exposure time corresponding to the sinusoidal phase shift stripes of different frequencies to shoot the sequence of pictures of the sinusoidal phase shift stripes of different frequencies, thereby avoiding the increase in the amount of data caused by multiple exposures of the sinusoidal phase shift stripes of each frequency, and saving computing resources. At the same time, only exposing the sinusoidal phase shift stripes of each frequency only once also simplifies the operation steps and reduces the difficulty of multi-frequency phase shift measurement.

[0095] In addition, since the camera exposure time is adjusted according to the frequency response characteristics of the projection optical system, more pixel intensity values ​​that can be used for phase calculation are retained under the same reflectivity. Therefore, it has higher robustness to high reflectivity. At the same time, compared with traditional methods, it requires smaller phase shift steps and has higher measurement efficiency.

[0096] Example 2

[0097] Based on the above embodiment, the following describes how a terminal device determines the optimal exposure time for a camera to capture multiple sinusoidal phase-shifted fringes of different frequencies. Figure 4 For a flow chart of another multi-frequency phase shift measurement method provided in an embodiment of the present application, see Figure 4 , the method comprises the following steps:

[0098] S301, controlling a projector to project a target object using a plurality of sinusoidal phase-shifted fringes of different frequencies.

[0099] The technical terms, technical effects, technical features, and optional implementation methods of step S301 can be found in Figure 3 It is understood that the step S201 shown is repeated and will not be described again here.

[0100] S302, determining an optimal exposure time for the camera to shoot the sinusoidal phase-shifted stripes of the first frequency according to an exposure area in a shooting picture of the sinusoidal phase-shifted stripes of the first frequency by the camera.

[0101] In some embodiments, after projecting sinusoidal phase-shift stripes of a first frequency, the camera can determine the exposure area in the captured image of the sinusoidal phase-shift stripes of the first frequency using multiple different exposure times, and use the exposure time corresponding to the most exposed area in the captured image of the sinusoidal phase-shift stripes of the first frequency as the optimal exposure time for the sinusoidal phase-shift stripes of the first frequency.

[0102] In this way, as many areas as possible in the picture taken by the camera can be accurately exposed, and thereby the optimal exposure time of the sinusoidal phase-shifted fringes of the first frequency can be obtained.

[0103] Preferably, the first frequency may be the highest frequency among a plurality of different frequencies.

[0104] S303, determining the optimal exposure time of the sinusoidal phase-shifted fringes excluding the first frequency according to the optimal exposure time of the sinusoidal phase-shifted fringes of the first frequency and the projection intensity ratio of a plurality of sinusoidal phase-shifted fringes of different frequencies on the reference plane.

[0105] In some embodiments, there is a corresponding relationship between the ratio of the optimal exposure time of the sinusoidal phase-shifted fringes and the ratio of the projection intensity of the sinusoidal phase-shifted fringes on the reference plane. According to the optimal exposure time of the sinusoidal phase-shifted fringes of the first frequency and the ratio of the projection intensity of the sinusoidal phase-shifted fringes of multiple different frequencies on the reference plane, the optimal exposure time of the sinusoidal phase-shifted fringes other than the first frequency can be determined.

[0106] For example, if the multiple sinusoidal phase shift strips of different frequencies include the frequency of the high-frequency sinusoidal phase shift strip The frequency of the intermediate frequency sinusoidal phase shift fringes and the frequency of the low-frequency sinusoidal phase-shifted fringes Its projection intensity ratio is P H :P M :P L When the frequency of the high-frequency sinusoidal phase shift strip is determined The optimal exposure time is T H After that, the projection intensity ratio can be used to determine the intermediate frequency The optimal exposure time is Low Frequency The optimal exposure time is

[0107] S304, obtaining a sequence of images of a plurality of sinusoidal phase-shifted fringes of different frequencies respectively captured by a camera using optimal exposure times of the sinusoidal phase-shifted fringes of different frequencies.

[0108] S305 , respectively calculating phase diagrams of a plurality of sinusoidal phase-shifted fringes of different frequencies according to a sequence of images of a plurality of sinusoidal phase-shifted fringes of different frequencies.

[0109] S306, determining three-dimensional information of the target object according to phase diagrams of a plurality of sinusoidal phase-shifted fringes of different frequencies.

[0110] For the technical terms, technical effects, technical features, and optional implementation methods of steps S304-S306, please refer to Figure 3 It is understood that the steps S203-S205 shown in the figure will not be repeated here for the repeated contents.

[0111] Example 3

[0112] Based on the above embodiment, the following describes how a terminal device calculates a phase diagram of a plurality of sinusoidal phase-shifted fringes of different frequencies. Figure 5 For a flow chart of another multi-frequency phase shift measurement method provided in the embodiment of the present application, please refer to Figure 5 , the method comprises the following steps:

[0113] S401, controlling a projector to project a target object using a plurality of sinusoidal phase-shifted fringes of different frequencies.

[0114] S402, determining the optimal exposure time for a camera to shoot a plurality of sinusoidal phase-shifted fringes with different frequencies according to the projection intensity ratio of the plurality of sinusoidal phase-shifted fringes with different frequencies on a reference plane.

[0115] S403, obtaining a sequence of images of a plurality of sinusoidal phase-shifted fringes of different frequencies respectively captured by a camera using optimal exposure times of the sinusoidal phase-shifted fringes of different frequencies.

[0116] For the technical terms, technical effects, technical features, and optional implementation methods of steps S401-S403, please refer to Figure 3 It is understood that the steps S201-S203 shown in the figure will not be repeated here for the repeated contents.

[0117] S404, determining the number of pixels at multiple pixel positions in a picture sequence of sinusoidal phase-shifted stripes of multiple different frequencies.

[0118] In some embodiments, for a plurality of pixel positions, the terminal device may determine the number of pixels at each pixel position in a sequence of images of sinusoidal phase-shifted stripes of a plurality of different frequencies.

[0119] Exemplarily, if there are 4 pictures in the picture sequence of sinusoidal phase-shifted stripes of the first frequency, the terminal device can determine whether there are pixels at the target pixel position in these 4 pictures, and record the number of pixels at the target pixel position in the picture sequence of sinusoidal phase-shifted stripes of the first frequency.

[0120] S405 , determining pixel labels corresponding to the plurality of pixel positions according to the number of pixels at the plurality of pixel positions in the image sequence of the sinusoidal phase-shifted fringes of the plurality of different frequencies.

[0121] It should be understood that the embodiments of the present application do not provide for how to determine pixel marks corresponding to multiple pixel positions. In some embodiments, the number of pixels and thresholds of multiple pixel positions in an image sequence of sinusoidal phase-shifted stripes of multiple different frequencies are compared to determine comparison results; based on the comparison results, pixel marks corresponding to the multiple pixel positions are determined.

[0122] Exemplarily, the terminal device may determine the comparison result of the number of pixels in the picture sequence of sinusoidal phase shift stripes of different frequencies and the threshold value from high frequency to low frequency. Taking the sinusoidal phase shift stripes including high frequency, medium frequency and low frequency as an example, the terminal device may determine the comparison result of the number of pixels in the picture sequence of sinusoidal phase shift stripes and the threshold value in the order of high frequency-medium frequency-low frequency. If the number of pixels in the picture sequence of high-frequency sinusoidal phase shift stripes is not less than the threshold value 3, the pixel position is marked as 1, and the comparison of the picture sequence of medium-frequency sinusoidal phase shift stripes and the picture sequence of low-frequency sinusoidal phase shift stripes is stopped. If the number of pixels in the picture sequence of high-frequency sinusoidal phase shift stripes is lower than the threshold value 3, it is determined whether the number of pixels in the picture sequence of medium-frequency sinusoidal phase shift stripes is lower than the threshold value 3. If the number of pixels in the picture sequence of medium-frequency sinusoidal phase shift stripes is not less than 3, the pixel position is marked as 2, and the comparison of the picture sequence of low-frequency sinusoidal phase shift stripes is stopped. If the number of pixels in the picture sequence of the intermediate frequency sinusoidal phase shift stripes is less than 3, it is determined whether the number of pixels in the picture sequence of the low frequency sinusoidal phase shift stripes is less than the threshold value 3. If the number of pixels in the picture sequence of the low frequency sinusoidal phase shift stripes is not less than 3, the pixel position is marked as 3, and if the number of pixels in the picture sequence of the low frequency sinusoidal phase shift stripes is less than 3, the pixel position is marked as 0.

[0123] It should be understood that the pixel data in the embodiment of the present application may be pixel data that is not overexposed.

[0124] S406, respectively calculating phase diagrams of a plurality of sinusoidal phase-shifted fringes of different frequencies according to the pixel labels corresponding to the plurality of pixel positions.

[0125] In some embodiments, there is a mapping relationship between the pixel labels corresponding to the plurality of pixel positions and the calculation method of the phase map.

[0126] Exemplarily, after traversing the pixel position, if the pixel corresponding to the pixel position is marked 1, the terminal device can calculate the phase map of the high frequency, medium frequency, and low frequency respectively. If the pixel corresponding to the pixel position is marked 2, the terminal device can calculate the phase map of the medium frequency and low frequency respectively. If the pixel corresponding to the pixel position is marked 3, the terminal device can calculate the phase map of the low frequency respectively. If the pixel corresponding to the pixel position is marked 0, the calculation of the pixel position is skipped.

[0127] It should be understood that the embodiments of the present application do not limit how to calculate the phase diagram of a plurality of sinusoidal phase-shifted fringes of different frequencies for each pixel position, and a commonly used method may be used.

[0128] S407, determining three-dimensional information of the target object according to the phase diagram of the sinusoidal phase-shifted fringes of multiple frequencies.

[0129] The technical terms, technical effects, technical features, and optional implementation methods of step S407 can be found in Figure 3 It is understood that the step S205 shown is repeated and will not be described again here.

[0130] Example 4

[0131] Based on the above embodiments, the following describes how a terminal device determines three-dimensional information of a target object based on a phase diagram of a plurality of sinusoidal phase-shifted fringes of different frequencies. Figure 6 For a flow chart of another multi-frequency phase shift measurement method provided in an embodiment of the present application, see Figure 6 , the method comprises the following steps:

[0132] S501, controlling a projector to project a target object using a plurality of sinusoidal phase-shifted fringes of different frequencies.

[0133] S502, determining the optimal exposure time for a camera to shoot a plurality of sinusoidal phase-shifted fringes with different frequencies according to the projection intensity ratio of the plurality of sinusoidal phase-shifted fringes with different frequencies on a reference plane.

[0134] S503, obtaining a sequence of images of a plurality of sinusoidal phase-shifted fringes of different frequencies respectively captured by a camera using optimal exposure times of the sinusoidal phase-shifted fringes of different frequencies.

[0135] S504, respectively calculating phase diagrams of a plurality of sinusoidal phase-shifted fringes of different frequencies according to a sequence of images of a plurality of sinusoidal phase-shifted fringes of different frequencies.

[0136] For the technical terms, technical effects, technical features, and optional implementation methods of steps S501-S504, please refer to Figure 3 It is understood that the steps S201-S204 shown are repeated and will not be described again here.

[0137] S505, unpacking the phase diagram of the sinusoidal phase shift fringes of the first frequency using the phase diagram of the sinusoidal phase shift fringes other than the first frequency to determine the three-dimensional information of the target object, wherein the first frequency is the highest frequency among the multiple different frequencies.

[0138] It should be understood that the unwrapping in the embodiment of the present application can use the low-frequency phase to unwrap the high-frequency phase to obtain its corresponding low-frequency equivalent phase.

[0139] For example, given the package phase The frequency ratio is first frequency f1: second frequency f2, f1>f2, then the terminal device can calculate the equivalent phase by formula (1).

[0140]

[0141] For example, if the target pixel is marked as 1, there are three wrapping phases, namely and First, it can be and Calculate the equivalent phase Then by and Calculate the equivalent phase, and the equivalent phase obtained is the absolute phase. If the pixel is marked as 2, there are two wrapped phases, which are and The absolute phase is calculated directly by formula (1). If the pixel is marked as 3, that is, only one phase This phase itself has no phase ambiguity and can therefore be used as an absolute phase.

[0142] In the embodiment of the present application, the depth information of the target object can be calculated using the absolute phase map and the position relationship between the camera and the projector. Finally, the depth information of the target object is combined with the X-axis and Y-axis coordinates of the pixels to obtain the three-dimensional information of the target object.

[0143] Example 5

[0144] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0145] See also Figure 7 , Figure 7 A schematic diagram of the structure of a multi-frequency phase shift measurement device provided in an embodiment of the present application. The multi-frequency phase shift measurement device can be implemented as all or part of an electronic device through software, hardware, or a combination of both. The device includes a projection module 601, an adjustment module 602, an acquisition module 603, and a calculation module 604.

[0146] Projection module 601, used to control the projector to project a target object using a plurality of sinusoidal phase-shifted fringes of different frequencies;

[0147] An adjustment module 602 is used to determine the optimal exposure time for the camera to shoot the multiple sinusoidal phase-shifted fringes with different frequencies according to the projection intensity ratio of the multiple sinusoidal phase-shifted fringes with different frequencies on the reference plane;

[0148] An acquisition module 603 is used to acquire a sequence of images of sinusoidal phase-shifted fringes of multiple frequencies respectively captured by a camera using optimal exposure times of sinusoidal phase-shifted fringes of multiple frequencies;

[0149] The calculation module 604 is used to calculate the phase diagrams of multiple sinusoidal phase shift fringes with different frequencies respectively according to the image sequences of multiple sinusoidal phase shift fringes with different frequencies; and determine the three-dimensional information of the target object according to the phase diagrams of multiple sinusoidal phase shift fringes with different frequencies.

[0150] Optionally, the adjustment module 602 is specifically used to determine the optimal exposure time of the camera for shooting sinusoidal phase-shifted stripes of the first frequency based on the exposure area in the shooting picture of the sinusoidal phase-shifted stripes of the first frequency by the camera; and determine the optimal exposure time of the sinusoidal phase-shifted stripes other than the first frequency based on the optimal exposure time of the sinusoidal phase-shifted stripes of the first frequency and the projection intensity ratio of multiple sinusoidal phase-shifted stripes of different frequencies on a reference plane.

[0151] Optionally, the first frequency is the highest frequency among multiple different frequencies.

[0152] Optionally, the adjustment module 602 is specifically used to determine the exposure area in the picture captured by the camera using multiple different exposure times for the sinusoidal phase-shift stripes of the first frequency; and use the exposure time corresponding to the most exposed area in the picture captured by the sinusoidal phase-shift stripes of the first frequency as the optimal exposure time for the sinusoidal phase-shift stripes of the first frequency.

[0153] Optionally, the calculation module 604 is specifically used to determine the number of pixels at multiple pixel positions in a picture sequence of sinusoidal phase-shifted stripes with multiple different frequencies; determine pixel marks corresponding to the multiple pixel positions based on the number of pixels at the multiple pixel positions in the picture sequence of sinusoidal phase-shifted stripes with multiple different frequencies; and calculate the phase diagrams of the sinusoidal phase-shifted stripes with multiple different frequencies respectively based on the pixel marks corresponding to the multiple pixel positions.

[0154] Optionally, the calculation module 604 is specifically used to compare the number of pixels at multiple pixel positions in the image sequence of sinusoidal phase-shifted stripes of multiple different frequencies with the threshold value to determine the comparison result;

[0155] According to the comparison result, pixel labels corresponding to the plurality of pixel positions are determined.

[0156] Optionally, the calculation module 604 is specifically configured to unpack the phase diagram of the sinusoidal phase shift fringes of the first frequency using the phase diagram of the sinusoidal phase shift fringes other than the first frequency to determine the three-dimensional information of the target object.

[0157] It should be noted that the multi-frequency phase shift measurement device provided in the above embodiment only uses the division of the above functional modules as an example when performing multi-frequency phase shift measurement. In practical applications, the above functional distribution can be completed by multiple different functional modules as needed, that is, the internal structure of the device is divided into multiple different functional modules to complete all or part of the functions described above. In addition, the multi-frequency phase shift measurement device provided in the above embodiment belongs to the same concept as the multi-frequency phase shift measurement embodiment. The implementation process thereof is detailed in the method embodiment and will not be repeated here.

[0158] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0159] With the multi-frequency phase shift measurement device, since the optimal exposure time is determined, the camera can directly use the optimal exposure time corresponding to the sinusoidal phase shift stripes of different frequencies to respectively shoot the image sequence of the sinusoidal phase shift stripes of different frequencies, thereby avoiding the increase in the amount of data caused by multiple exposures of the sinusoidal phase shift stripes of each frequency, saving computing resources. At the same time, only exposing the sinusoidal phase shift stripes of each frequency once also simplifies the operation steps and reduces the difficulty of multi-frequency phase shift measurement.

[0160] Example 6

[0161] See also Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device 1000 may include: at least one processor 1001 , at least one network interface 1004 , a user interface 1003 , a memory 1005 , and at least one communication bus 1002 .

[0162] The communication bus 1002 is used to realize the connection and communication between these components.

[0163] The user interface 1003 may include a display screen (Display) and a camera (Camera), and the optional user interface 1003 may also include a standard wired interface and a wireless interface.

[0164] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0165] Among them, the processor 1001 may include one or more processing cores. The processor 1001 uses various interfaces and lines to connect various parts within the entire electronic device 1000, and executes various functions and processes data of the electronic device 1000 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1005, and calling data stored in the memory 1005. Optionally, the processor 1001 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 1001 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 1001, and it can be implemented separately through a chip.

[0166] Among them, the memory 1005 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 1005 may optionally be at least one storage device located away from the aforementioned processor 1001. As Figure 8 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an operating application of the electronic device.

[0167] exist Figure 8In the electronic device 1000 shown, the user interface 1003 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 1001 can be used to call the operating application of the electronic device stored in the memory 1005, and specifically perform the following operations:

[0168] Controlling the projector to project a plurality of sinusoidal phase-shifted fringes of different frequencies onto the target object;

[0169] According to the projection intensity ratio of the multiple sinusoidal phase-shifted fringes with different frequencies on the reference plane, the optimal exposure time for the camera to shoot the multiple sinusoidal phase-shifted fringes with different frequencies is determined respectively;

[0170] Acquire a sequence of images of a plurality of sinusoidal phase-shifted fringes of different frequencies taken by a camera using the optimal exposure time of the sinusoidal phase-shifted fringes of different frequencies;

[0171] According to a sequence of images of a plurality of sinusoidal phase-shifted fringes of different frequencies, phase diagrams of the plurality of sinusoidal phase-shifted fringes of different frequencies are calculated respectively;

[0172] The three-dimensional information of the target object is determined based on the phase diagram of multiple sinusoidal phase-shifted fringes of different frequencies.

[0173] In one embodiment, the processor 1001 further performs the following steps:

[0174] Determining an optimal exposure time for the camera to shoot the sinusoidal phase-shifted stripes of the first frequency according to an exposure area in a shooting picture of the sinusoidal phase-shifted stripes of the first frequency by the camera;

[0175] The optimal exposure time of the sinusoidal phase-shifted fringes except the first frequency is determined according to the optimal exposure time of the sinusoidal phase-shifted fringes of the first frequency and the projection intensity ratio of a plurality of sinusoidal phase-shifted fringes of different frequencies on the reference plane.

[0176] In one embodiment, the first frequency is a highest frequency among the plurality of different frequencies.

[0177] In one embodiment, the processor 1001 further performs the following steps:

[0178] Determine an exposure area in a picture captured by a camera using a plurality of different exposure times for sinusoidal phase-shifted stripes of a first frequency;

[0179] The exposure time corresponding to the most exposed area in the shooting picture of the sinusoidal phase-shifted stripes of the first frequency is used as the optimal exposure time of the sinusoidal phase-shifted stripes of the first frequency.

[0180] In one embodiment, the processor 1001 further performs the following steps:

[0181] Determining the number of pixels at a plurality of pixel positions in a sequence of images of sinusoidal phase-shifted fringes of a plurality of different frequencies;

[0182] Determine pixel labels corresponding to the plurality of pixel positions according to the number of pixels at the plurality of pixel positions in a sequence of images of sinusoidal phase-shifted fringes of a plurality of different frequencies;

[0183] Phase diagrams of a plurality of sinusoidal phase-shifted fringes of different frequencies are calculated respectively according to the pixel marks corresponding to the plurality of pixel positions.

[0184] In one embodiment, the processor 1001 further performs the following steps:

[0185] Compare the number of pixels at multiple pixel positions in a picture sequence of sinusoidal phase-shifted stripes of multiple different frequencies with a threshold value, respectively, and determine a comparison result;

[0186] According to the comparison result, pixel labels corresponding to the plurality of pixel positions are determined.

[0187] In one embodiment, the processor 1001 further performs the following steps:

[0188] The phase diagram of the sinusoidal phase shift fringes of the first frequency is unwrapped using the phase diagram of the sinusoidal phase shift fringes other than the first frequency to determine the three-dimensional information of the target object.

[0189] With the electronic device, since the optimal exposure time is determined, the camera can directly use the optimal exposure time corresponding to the sinusoidal phase shift stripes of different frequencies to respectively shoot the image sequence of the sinusoidal phase shift stripes of different frequencies, thereby avoiding the increase of the data volume caused by multiple exposures of the sinusoidal phase shift stripes of each frequency, saving computing resources. At the same time, only exposing the sinusoidal phase shift stripes of each frequency once also simplifies the operation steps and reduces the difficulty of multi-frequency phase shift measurement.

[0190] The present application also provides a computer storage medium that can store multiple instructions, which are suitable for being loaded and executed by a processor as described above. Figure 3-Figure 6 The method steps of the embodiment shown in the figure can be found in the specific implementation process. Figure 3-Figure 6 The specific description of the illustrated embodiment will not be repeated here.

[0191] The device where the storage medium is located may be the above-mentioned electronic device.

[0192] An embodiment of the present application provides a computer program product, including computer instructions, which implement the above method when executed by a processor.

[0193] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0194] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0195] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0196] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0197] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0198] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0199] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0200] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0201] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A multi-frequency phase shift measurement method, characterized in that: include: Controlling the projector to project a plurality of sinusoidal phase-shifted fringes of different frequencies onto the target object; Determining respectively the optimal exposure time for a camera to shoot the multiple sinusoidal phase-shifted fringes of different frequencies according to the projection intensity ratios of the multiple sinusoidal phase-shifted fringes of different frequencies on the reference plane; Acquire a sequence of images of the plurality of sinusoidal phase-shifted fringes with different frequencies respectively captured by the camera using the optimal exposure time of the plurality of sinusoidal phase-shifted fringes with different frequencies; Calculating phase diagrams of the plurality of sinusoidal phase-shifted fringes of different frequencies respectively according to the image sequences of the plurality of sinusoidal phase-shifted fringes of different frequencies; Determining three-dimensional information of the target object according to a phase diagram of the plurality of sinusoidal phase-shifted fringes of different frequencies; The method of determining the optimal exposure time for a camera to shoot the plurality of sinusoidal phase-shifted fringes with different frequencies respectively according to the projection intensity ratio of the plurality of sinusoidal phase-shifted fringes with different frequencies on the reference plane comprises: Determine, according to the exposure area in the shooting picture of the sinusoidal phase-shifted stripes of the first frequency by the camera, the optimal exposure time for the camera to shoot the sinusoidal phase-shifted stripes of the first frequency, wherein the optimal exposure time for the sinusoidal phase-shifted stripes of the first frequency is the exposure time corresponding to the most exposure area in the shooting picture of the sinusoidal phase-shifted stripes of the first frequency; The optimal exposure time of the sinusoidal phase-shifted fringes other than the first frequency is determined according to the optimal exposure time of the sinusoidal phase-shifted fringes of the first frequency and the projection intensity ratio of the multiple sinusoidal phase-shifted fringes of different frequencies on the reference plane.

2. The method according to claim 1, characterized in that The first frequency is a highest frequency among the multiple different frequencies.

3. The method according to claim 1 or 2, characterized in that: The step of determining the optimal exposure time for the camera to shoot the sinusoidal phase-shifted fringes of the first frequency according to the exposure area in the shooting picture of the camera shooting the sinusoidal phase-shifted fringes of the first frequency comprises: Determine an exposure area in a picture captured by the camera using a plurality of different exposure times for the sinusoidal phase-shifted fringes of the first frequency; The exposure time corresponding to the most exposed area in the shooting picture of the sinusoidal phase-shifted stripes of the first frequency is used as the optimal exposure time of the sinusoidal phase-shifted stripes of the first frequency.

4. The method according to claim 1, characterized in that: The step of respectively calculating the phase diagrams of the plurality of sinusoidal phase-shifted fringes with different frequencies according to the image sequences of the plurality of sinusoidal phase-shifted fringes with different frequencies comprises: Determine the number of pixels at a plurality of pixel positions in a sequence of images of the plurality of sinusoidal phase-shifted stripes of different frequencies; Determine pixel labels corresponding to the plurality of pixel positions according to the number of pixels of the plurality of pixel positions in the image sequence of the plurality of sinusoidal phase-shifted fringes of different frequencies; Phase diagrams of the multiple sinusoidal phase-shifted fringes of different frequencies are calculated respectively according to the pixel marks corresponding to the multiple pixel positions.

5. The method according to claim 4, characterized in that The step of determining pixel labels corresponding to the plurality of pixel positions according to the number of pixels of the plurality of pixel positions in the image sequence of the plurality of sinusoidal phase-shifted fringes of different frequencies comprises: Compare the number of pixels of the plurality of pixel positions in the image sequence of the plurality of sinusoidal phase-shifted fringes of different frequencies with a threshold value respectively, and determine a comparison result; According to the comparison result, pixel labels corresponding to the plurality of pixel positions are determined.

6. The method according to claim 1, characterized in that Determining the three-dimensional information of the target object according to the phase diagram of the plurality of sinusoidal phase-shifted fringes of different frequencies comprises: The phase diagram of the sinusoidal phase shift fringes other than the first frequency is used to unwrap the phase diagram of the sinusoidal phase shift fringes of the first frequency to determine the three-dimensional information of the target object.

7. A multi-frequency phase shift measurement device, characterized in that: include: A projection module, used to control a projector to project a target object using a plurality of sinusoidal phase-shifted fringes of different frequencies; An adjustment module, used to determine the optimal exposure time for a camera to shoot the multiple sinusoidal phase-shifted fringes with different frequencies, respectively, according to the projection intensity ratio of the multiple sinusoidal phase-shifted fringes with different frequencies on the reference plane; An acquisition module, used for acquiring a sequence of images of the plurality of sinusoidal phase-shifted fringes with different frequencies respectively captured by the camera using the optimal exposure time of the plurality of sinusoidal phase-shifted fringes with different frequencies; A calculation module, configured to calculate phase diagrams of the plurality of sinusoidal phase-shifted fringes with different frequencies respectively according to the image sequences of the plurality of sinusoidal phase-shifted fringes with different frequencies; and determine the three-dimensional information of the target object according to the phase diagrams of the plurality of sinusoidal phase-shifted fringes with different frequencies; The adjustment module is specifically used to determine the optimal exposure time of the camera for shooting the sinusoidal phase-shifted stripes of the first frequency according to the exposure area in the shooting picture of the sinusoidal phase-shifted stripes of the first frequency by the camera, wherein the optimal exposure time of the sinusoidal phase-shifted stripes of the first frequency is the exposure time corresponding to the largest exposure area in the shooting picture of the sinusoidal phase-shifted stripes of the first frequency; and determine the optimal exposure time of the sinusoidal phase-shifted stripes other than the first frequency according to the optimal exposure time of the sinusoidal phase-shifted stripes of the first frequency and the projection intensity ratio of the multiple sinusoidal phase-shifted stripes of different frequencies on a reference plane.

8. The device according to claim 7, characterized in that The first frequency is a highest frequency among the multiple different frequencies.

9. The device according to claim 7 or 8, characterized in that The adjustment module is specifically used to determine the exposure area in the shooting picture of the sinusoidal phase-shift stripes of the first frequency using multiple different exposure times by the camera; and use the exposure time corresponding to the most exposed area in the shooting picture of the sinusoidal phase-shift stripes of the first frequency as the optimal exposure time of the sinusoidal phase-shift stripes of the first frequency.

10. The device according to claim 9, characterized in that The calculation module is specifically used to determine the number of pixels at multiple pixel positions in the image sequence of the multiple sinusoidal phase-shifted stripes with different frequencies; and determine the pixel labels corresponding to the multiple pixel positions according to the number of pixels at the multiple pixel positions in the image sequence of the multiple sinusoidal phase-shifted stripes with different frequencies; Phase diagrams of the multiple sinusoidal phase-shifted fringes of different frequencies are calculated respectively according to the pixel marks corresponding to the multiple pixel positions.

11. The device according to claim 10, characterized in that The calculation module is specifically used to compare the number of pixels of the multiple pixel positions in the image sequence of the multiple sinusoidal phase-shifted fringes with different frequencies with the threshold value, and determine the comparison result; According to the comparison result, pixel labels corresponding to the plurality of pixel positions are determined.

12. The device according to claim 7, characterized in that The calculation module is specifically used to unpack the phase diagram of the sinusoidal phase shift fringes of the first frequency using the phase diagram of the sinusoidal phase shift fringes other than the first frequency to determine the three-dimensional information of the target object.

13. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

14. A computer storage medium, characterized in that: The computer storage medium stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 6.

15. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 6.

Images