An image generation system and an image generation method

Abstract

The present specification discloses an image generation system and an image generation method, which can obtain natural light reflected by a target object through an imaging lens, and then filter out part of the obtained light rays containing image information corresponding to a specified light source point arranged on the target object (i.e. light rays in a specified wave band) through a light splitting piece to obtain first light rays and second light rays, and then obtain a first image corresponding to the specified light source point according to the first light rays, determine the influence degree of the light propagation medium on the collected image of the target object according to the diffusion degree of the halo in the first image, and restore the image of the target object disturbed by the light propagation medium to generate an image of the target object before being disturbed by the light propagation medium, so that the cost of generating the image of the target object can be reduced while improving the definition of the generated image of the target object.

Description

Technical Field

[0001] This specification relates to the field of photoelectric imaging technology, and in particular to an image generation system and an image generation method. Background Technology

[0002] Currently, in fields such as aerospace and astronomical research, it is often necessary to obtain images of targets at high altitudes from the ground or to obtain images of targets on the ground from high altitudes in order to perform corresponding tasks based on the obtained images. For example, when guiding an aircraft to land, it is necessary to obtain an image of the aircraft's fuselage so that the aircraft can be guided to land based on the image of the aircraft's fuselage.

[0003] However, when acquiring images of the target object, the image may be low in clarity due to complex environmental factors (such as atmospheric turbulence, haze, and smoke from forest fires), making it unusable for subsequent tasks.

[0004] Existing technologies model each medium based on its transmission characteristics, which can interfere with image acquisition in different environments, in order to perform image processing based on the transmission characteristics of different media. However, in practical applications, the cost of accurately modeling the transmission characteristics of media in different environments is relatively high.

[0005] Therefore, how to reduce the cost of generating images of target objects in complex environments is an urgent problem to be solved. Summary of the Invention

[0006] This specification provides an image generation system and an image generation method to partially solve the aforementioned problems existing in the prior art.

[0007] The following technical solution is adopted in this specification:

[0008] This specification provides an image generation system, which includes: an optical imaging unit and an image processing unit. The optical imaging unit includes: an imaging lens and a beam splitter.

[0009] The imaging lens is used to acquire light from the target object and transmit the light to the beam splitter;

[0010] The beam splitter is used to filter the light rays, filtering out light rays in a specified wavelength band as the first light ray, and filtering out light rays in other wavelength bands besides the first light ray as the second light ray. The first light signal corresponding to the first light ray and the second light signal corresponding to the second light ray are transmitted to the image processing unit. The first light signal is used to represent the image information of the image generation system acquired by the image generation system at a specified light source point on the target object under the condition of interference from the light propagation medium. The second light signal is used to represent the image information of the image generation system acquired by the image generation system that does not contain the light emitted by the specified light source point under the condition of interference from the light propagation medium.

[0011] The image processing unit is used to obtain a first image based on the first light signal, wherein the first image is an image diffused from the designated light source point under the condition of interference from the light propagation medium;

[0012] Based on the first image, the second image obtained based on the second light signal is restored to generate an image of the target object before it was interfered with by the light propagation medium.

[0013] Optionally, the optical imaging unit further includes: a first camera and a second camera, wherein the first camera and the second camera are respectively disposed in two different directions of the beam splitter;

[0014] The first camera is used to receive the first light ray transmitted by the beam splitter, generate a first light signal corresponding to the first light ray based on the first light ray, and transmit the first light signal to the image processing unit.

[0015] The second camera is used to receive the second light ray transmitted by the beam splitter, generate a second light signal corresponding to the second light ray based on the second light ray, and transmit the second light signal to the image processing unit.

[0016] Optionally, the optical imaging unit further includes: a first filter and a second filter, wherein the first filter is disposed between the first camera and the beam splitter, and the second filter is disposed between the second camera and the beam splitter;

[0017] The first filter is used to filter the first light to obtain filtered first light, and then transmits the filtered first light to the first camera;

[0018] The second filter is used to filter the second light to obtain filtered second light, and then transmits the filtered second light to the second camera.

[0019] Optionally, the image processing unit is configured to perform a Fourier transform on the first image to obtain the value of the first image after the Fourier transform; and

[0020] Perform a Fourier transform on the second image to obtain the value of the second image after the Fourier transform;

[0021] Based on the Fourier transform values ​​of the first image and the second image, determine the Fourier transform values ​​corresponding to the image of the target object before it is interfered with by the light propagation medium.

[0022] Perform an inverse Fourier transform on the value corresponding to the image of the target object before it is interfered with by the light propagation medium, and obtain the image of the target object before it is interfered with by the light propagation medium.

[0023] Optionally, the image processing unit is used to input the first image and the second image into a pre-calibrated restoration model, so as to restore the second image based on the first image through the restoration model, so as to obtain an image of the target object before it is interfered with by the light propagation medium.

[0024] Optionally, the image generation system further includes: a calibration unit;

[0025] The calibration unit is used to input the first image sample, which is collected and set at a specified light source point on the sample target, and the second image sample of the sample target into the restoration model, so that the restoration model can restore the second image sample based on the first image sample to obtain the image of the sample target before it is interfered with by the light propagation medium.

[0026] The calibration target is to minimize the deviation between the generated image of the target sample before it is affected by the light propagation medium and the actual image of the target sample. The parameters in the restoration model are then calibrated to obtain the calibrated restoration model.

[0027] This specification provides an image generation method applied to an image processing system. The image generation system includes an optical imaging unit and an image processing unit. The optical imaging unit includes an imaging lens and a beam splitter. The method includes:

[0028] The image processing unit receives a first optical signal and a second optical signal. The first optical signal is obtained by filtering the light from the target object using the beam splitter to filter out the first light rays in a specified wavelength band. The second optical signal is obtained by converting the second light rays in other wavelength bands besides the first light rays. The first optical signal is used to represent image information of a specified light source point on the target object acquired by the image generation system when the light propagation medium interferes with the image. The second optical signal is used to represent image information of the image generation system that does not contain the light emitted by the specified light source point when the light propagation medium interferes with the image. The light from the target object is acquired through the imaging lens.

[0029] Based on the first light signal, a first image is obtained, wherein the first image is the image diffused from the designated light source point under the condition of interference from the light propagation medium;

[0030] Based on the first image, the second image obtained based on the second light signal is restored to generate an image of the target object before it was interfered with by the light propagation medium.

[0031] Optionally, the optical imaging unit further includes: a first camera and a second camera, wherein the first camera and the second camera are respectively disposed in two different directions of the beam splitter;

[0032] The process of filtering out the first ray of light in a specified wavelength band and converting it to obtain a first optical signal specifically includes:

[0033] The beam splitter filters out light rays in a specified wavelength range, which are then used as first light rays. The first light rays are transmitted to the first camera, which generates a first light signal corresponding to the first light rays based on the first light rays. The first light signal is then transmitted to the image processing unit.

[0034] Converting second light rays of other wavelengths besides the first light ray into a second optical signal specifically includes:

[0035] Light of other wavelengths besides the first light is taken as the second light and transmitted to the second camera. The second camera generates a second light signal corresponding to the second light based on the second light and transmits the second light signal to the image processing unit.

[0036] Optionally, the optical imaging unit further includes: a first filter and a second filter, wherein the first filter is disposed between the first camera and the beam splitter, and the second filter is disposed between the second camera and the beam splitter;

[0037] Transmitting the first light beam to the first camera specifically includes:

[0038] The first light is filtered by the first filter to obtain the filtered first light, and the filtered first light is transmitted to the first camera.

[0039] Transmitting the second light beam to the second camera specifically includes:

[0040] The second light is filtered by the second filter to obtain the filtered second light, and the filtered second light is transmitted to the second camera.

[0041] Optionally, based on the first image, a second image obtained from the second optical signal is restored to generate an image of the target object before it is interfered with by the light propagation medium, specifically including:

[0042] Perform a Fourier transform on the first image to obtain the value of the first image after the Fourier transform; and

[0043] Perform a Fourier transform on the second image to obtain the value of the second image after the Fourier transform;

[0044] Based on the Fourier transform values ​​of the first image and the second image, determine the Fourier transform values ​​corresponding to the image of the target object before it is interfered with by the light propagation medium.

[0045] Perform an inverse Fourier transform on the value corresponding to the processed image after Fourier transform to obtain the image of the target object before it is interfered with by the light propagation medium.

[0046] Optionally, based on the first image, the second image obtained from the second optical signal is restored, specifically including:

[0047] The first image and the second image are input into a pre-calibrated restoration model, so that the second image is restored based on the first image by the restoration model, so as to obtain the image of the target object before it is interfered with by the light propagation medium.

[0048] Optionally, the image generation system further includes: a calibration unit;

[0049] The calibration of the restoration model specifically includes:

[0050] The first image sample, which is set at a specified light source point on the target object, and the second image sample of the target object are collected and input into the restoration model. The restoration model restores the second image sample based on the first image sample to obtain the image of the target object before it is interfered with by the light propagation medium.

[0051] The calibration target is to minimize the deviation between the generated image of the target sample before it is affected by the light propagation medium and the actual image of the target sample. The parameters in the restoration model are then calibrated to obtain the calibrated restoration model.

[0052] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described image generation method.

[0053] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image generation method described above.

[0054] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0055] In the image generation method provided in this specification, the image processing unit receives a first optical signal and a second optical signal. The first optical signal is obtained by filtering the light from the target object using a beam splitter to filter out the first light rays in a specified wavelength band and then converting them. The second optical signal is obtained by converting the second light rays in other wavelength bands besides the first light rays. The first optical signal is used to represent the image information of a specified light source point on the target object acquired by the image generation system when the light propagation medium interferes with the image. The second optical signal is used to represent the image information of the image generation system that does not contain the light emitted by the specified light source point when the light propagation medium interferes with the image. The light from the target object is acquired through an imaging lens. Based on the first optical signal, a first image is obtained, wherein the first image is the image diffused from the specified light source point when the light propagation medium interferes with the image. Based on the first image, the second image obtained based on the second optical signal is restored to generate an image of the target object before the light propagation medium interferes with the image.

[0056] As can be seen from the above method, the natural light reflected back by the target object can be acquired through the imaging lens. Then, a beam splitter can be used to filter out the portion of the acquired light that contains image information corresponding to the specified light source point set on the target object (i.e., light in the specified wavelength band), which is used as the first light ray, and the remaining light ray is used as the second light ray. Then, a first image corresponding to the specified light source point can be generated based on the first light ray. Since the first image is the image of the specified light source point under the condition of interference from the light propagation medium, if the first image is not affected by the light propagation medium, the acquired first image should be a light spot. However, after the first image is affected by the light propagation medium, the acquired first image is a halo diffused from the light spot. Therefore, the degree of diffusion of the halo in the first image can be used to determine the degree of influence of the light propagation medium on the acquired image of the target object. Based on this, the image of the target object affected by the interference can be restored to generate the image of the target object before the interference of the light propagation medium. Thus, the clarity of the generated image of the target object can be improved while reducing the cost of generating the image of the target object. Attached Figure Description

[0057] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0058] Figure 1 This is a schematic diagram of an image generation system provided in this specification;

[0059] Figure 2 This is a schematic diagram of the optical path of the first and second rays provided in this specification;

[0060] Figure 3 This is a schematic diagram of different image regions corresponding to the specified light source provided in this specification.

[0061] Figure 4 This is a schematic diagram of an image generation method provided in this specification;

[0062] Figure 5 This specification provides a corresponding Figure 1 A schematic diagram of an electronic device. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0064] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0065] Currently, in fields such as aerospace and astronomical research, it is often necessary to obtain images of targets at high altitudes from the ground or to obtain images of targets on the ground from high altitudes in order to perform corresponding tasks based on the obtained images. For example, when guiding an aircraft to land, it is necessary to obtain an image of the aircraft's fuselage so that the aircraft can be guided to land based on the image of the aircraft's fuselage.

[0066] However, when acquiring images of a target object, the clarity of the acquired image may be low due to the influence of complex environments (such as atmospheric turbulence, haze, and smoke from forest fires). In order to improve the clarity of the generated target object image, it is necessary to model the media that interfere with the acquisition of the target object image in different environments. This method has low adaptability to different complex environments and requires high cost.

[0067] Based on this, this specification provides an image generation system, such as... Figure 1 As shown:

[0068] Figure 1 This is a schematic diagram of an image generation system provided in this specification. Figure 1 As can be seen, the image generation system includes: an optical imaging unit and an image processing unit. The optical imaging unit includes: an imaging lens and a beam splitter.

[0069] Specifically, the aforementioned imaging lens is used to acquire the light from the target object and transmit the acquired light to the beam splitter. Here, the light from the target object refers to the light emitted by the imaging lens after the natural light shines on the surface of the target object and is reflected by the target object, as well as the light emitted by the designated light source point set on the target object.

[0070] The designated light source point can refer to a light source device that is pre-installed on the target object. For example, if the target object is an airplane, the designated light source point can be the navigation light on the airplane. As another example, if the target object is a lighthouse, the designated light source point can be the searchlight installed on the lighthouse.

[0071] The aforementioned beam splitter is used to filter the light acquired by the imaging lens, so as to filter out the light in a specified wavelength band as the first light, and to filter out the light in other wavelength bands besides the first light as the second light.

[0072] The beam splitter can be a partial bandpass cutoff filter. This type of beam splitter can have different reflectivities for light in different wavelength bands. For example, it can make the average reflectivity of light below 600nm > 90%, while the transmittance of light above 610nm > 90%. Therefore, it can allow the first light to pass through the beam splitter and be filtered out, and can allow the second light to be reflected.

[0073] Furthermore, a first camera and a second camera are also provided on two different directions of the beam splitter. The first light rays can be collected by the first camera after passing through the beam splitter, and the second light rays can be collected by the second camera after being reflected by the beam splitter, specifically as follows: Figure 2 As shown.

[0074] Figure 2 This is a schematic diagram of the optical path of the first and second rays provided in this specification.

[0075] from Figure 2 As can be seen, a first filter is provided between the first camera and the beam splitter, and a second filter is provided between the second camera and the beam splitter. The first filter and the second filter can be selected according to actual needs. For example, the first filter can be a narrowband filter. The narrowband filter can make the first light in the first specified wavelength range completely pass through and be collected by the first camera. The first specified wavelength range can be a center wavelength of 650nm and a spectral half-width of ±10nm.

[0076] The second filter mentioned above can be a broadband filter, which allows the second light rays within the second specified wavelength range to pass through completely and be collected by the second camera. The second specified wavelength range can be a center wavelength of 550nm and a spectral half-width of ±50nm.

[0077] It should be noted that the first and second filters mentioned above can filter out light from other wavelengths to avoid interference from light signals from other wavelengths on the first or second light, thereby increasing the clarity of the generated target image.

[0078] Furthermore, after the first camera acquires the first ray, it can generate a first light signal based on the first ray and transmit the first light signal to the image processing unit. After the second camera acquires the second ray, it can generate a second light signal corresponding to the second ray and transmit the second light signal to the image processing unit. The first light signal is used to represent the image information acquired by the image generation system at a specified light source point on the target object when the light propagation medium interferes with the image. The second light signal is used to represent the image information acquired by the image generation system that does not contain the light emitted by the specified light source point when the light propagation medium interferes with the image.

[0079] In this specification, the image processing unit can obtain a first image based on a first light signal, and determine the image area diffused in the first image when a specified light source point is interfered with by a light propagation medium. Based on the first image, it restores the second image obtained based on a second light signal to generate an image of the target object before it is interfered with by the light propagation medium. Here, the light propagation medium can refer to media such as haze or atmospheric turbulence. The restoration of the second image obtained based on the second light signal from the constructed first image can be referred to the following formula:

[0080] g(x,y)=h(x,y)*f(x,y)+eta(x,y)

[0081] Where g(x,y) is the second image, that is, the image of the target object under the interference of the light propagation medium, h(x,y) is the first image, f(x,y) is the image of the target object before the light propagation medium interferes, and η(x,y) is noise.

[0082] In practical applications, since the above η(x,y) often cannot be accurately quantified, the image processing unit can also transform the above formula through algorithms such as Wiener filtering to obtain a transformed formula, and then restore the second image obtained based on the second light signal based on the transformed formula.

[0083] Specifically, the image processing unit can perform a Fourier transform on the first image to obtain the Fourier transformed value of the first image, and can also perform a Fourier transform on the second image to obtain the Fourier transformed value of the second image. Based on the Fourier transformed values ​​of the first and second images, the Fourier transformed value corresponding to the image of the target object before it is interfered with by the light propagation medium is determined. An inverse Fourier transform is then performed on the Fourier transformed value corresponding to the processed image to obtain the image of the target object before it is interfered with by the light propagation medium. The specific details can be found in the following formula:

[0084]

[0085] In the above formula, H(u,v) represents the value of the processed image of the target object after Fourier transform, G(u,v) represents the value of the first image after Fourier transform, and K is a pre-defined non-zero constant less than 1.

[0086] It should be noted that a transmission device may also be provided between the first camera and the image processing unit, and between the second camera and the image processing unit, so that the first optical signal and the second optical signal can be transmitted to the image processing unit through the transmission device. The transmission device here includes one of the following: coaxial cable and serial bus.

[0087] To illustrate the above in detail, this specification also provides a schematic diagram of the first image obtained when a specified light source point is interfered with by the light propagation medium, such as... Figure 3 As shown.

[0088] from Figure 3 As can be seen from the left image, if the specified light source point is not interfered with by the light propagation medium, the first image should be a single light point. Figure 3 As can be seen in the image on the right, the first image when the specified light source point is interfered with by the light propagation medium is a halo. Therefore, the degree of diffusion of the halo in the first image can be determined based on the image information of the first image. It can be understood that the first image before the specified light source point is interfered with by the light propagation medium is taken as the standard image, and the degree of influence of the light propagation medium on the image of the target object can be determined based on the difference between the first image after the specified light source point is interfered with by the light propagation medium and the above-mentioned standard image. Then, the second image of the target object after being interfered with by the light propagation medium can be restored.

[0089] Image information can refer to the location of each pixel in the image, as well as information such as pixel value.

[0090] In addition, the image processing unit can also input the first image and the second image into a pre-trained restoration model, so that the restoration model can restore the second image based on the first image to obtain the image of the target object before it is interfered with by the light propagation medium.

[0091] Of course, before deploying the above-mentioned restoration model to the server, the parameters of the restoration model need to be calibrated. Based on this, the image generation system also includes a calibration unit, which is used to input the first image sample, which is set at a specified light source point on the target object, and the second image sample of the target object, into the restoration model. The restoration model then uses the first image sample to restore the second image sample to obtain the image of the target object before it is affected by the light propagation medium. The calibration target is to minimize the deviation between the generated image of the target object before it is affected by the light propagation medium and the actual image of the target object. The parameters in the restoration model are calibrated to obtain the calibrated restoration model. For details, please refer to the following formula:

[0092]

[0093] In the above formula, e 2 Let f be the mean square error between the image of the target object before it is affected by the light propagation medium, obtained through the restoration model, and the actual image of the target object. Let E be the image of the sample target object before it is affected by the light propagation medium, obtained through the restoration model, where E|·| is the expected value of the argument.

[0094] As can be seen from the above, the natural light reflected back by the target object can be acquired through an imaging lens. A beam splitter can then be used to filter out the portion of the acquired light containing image information corresponding to a designated light source point on the target object (i.e., light in a designated wavelength band), which serves as the first ray. The remaining light is used as the second ray. A first image corresponding to the designated light source point can then be generated based on the first ray. Since the first image is the image of the designated light source point under conditions of light propagation medium interference, if the first image is not interfered with by the light propagation medium, the acquired first image should be a single light point. However, if the first image is interfered with by the light propagation medium, the acquired first image is a halo diffused from the aforementioned light point. Therefore, the degree of halo diffusion in the first image can determine the degree of influence of the light propagation medium on the acquired image of the target object. Based on this, the image of the target object affected by the interference can be restored, generating the image of the target object before the interference by the light propagation medium. This can improve the clarity of the generated image of the target object while reducing the cost of generating the image.

[0095] In addition, since the optical acquisition units that generate the first and second images are relatively simple, the adaptability of the image generation system in various complex environments can be improved.

[0096] To further illustrate the image generation system described above, this specification also provides a method for image processing using the image generation system, specifically as follows: Figure 4 As shown.

[0097] Figure 4 This is a schematic diagram of an image generation method provided in this specification, including the following steps:

[0098] S401: The image processing unit receives a first optical signal and a second optical signal. The first optical signal is obtained by filtering the light from the target object using the beam splitter to filter out the first light rays in a specified wavelength band. The second optical signal is obtained by converting the second light rays in other wavelength bands besides the first light rays. The first optical signal is used to represent the image information of the target object set at a specified light source point by the image generation system when the light propagation medium interferes with the image. The second optical signal is used to represent the image information of the target object that does not contain the light emitted by the specified light source point by the image generation system when the light propagation medium interferes with the image. The light from the target object is obtained through the imaging lens.

[0099] In this specification, the image processing unit can receive a first optical signal and a second optical signal. The first optical signal is obtained by filtering the light from the target object using the beam splitter to filter out the first light rays in a specified wavelength band. The second optical signal is obtained by converting the second light rays in other wavelength bands besides the first light rays. The first optical signal is used to represent image information of a specified light source point on the target object acquired by the image generation system when the light propagation medium interferes with the image. The second optical signal is used to represent image information of the image generation system that does not contain the light emitted by the specified light source point when the light propagation medium interferes with the image. The light from the target object is acquired through the imaging lens.

[0100] In this specification, the execution subject used to implement the image generation method can refer to a device such as a laptop computer or a desktop computer, or it can refer to a specific device set in the image generation system. For ease of description, the image generation method provided in this specification will be described below using the image processing unit as the execution subject as an example.

[0101] S402: Based on the first light signal, a first image is obtained, wherein the first image is the image diffused from the designated light source point under the condition of interference from the light propagation medium.

[0102] S403: Based on the first image, restore the second image obtained based on the second light signal to generate an image of the target object before it is interfered with by the light propagation medium.

[0103] Based on the first light signal, a first image is obtained, which is the image diffused when the specified light source point is interfered with by the light propagation medium; based on the first image, a second image obtained based on the second light signal is restored to generate an image of the target object before it is interfered with by the light propagation medium.

[0104] Specifically, the optical imaging unit further includes a first camera and a second camera, wherein the first camera and the second camera are respectively disposed in two different directions of the beam splitter. The image processing unit can filter out light rays in a specified wavelength range from the light rays through the beam splitter as first light rays and transmit the first light rays to the first camera, so that the first camera can generate a first light signal corresponding to the first light rays and transmit the first light signal to the image processing unit. The unit can also use light rays in other wavelength ranges besides the first light rays as second light rays and transmit the second light rays to the second camera, so that the second camera can generate a second light signal corresponding to the second light rays and transmit the second light signal to the image processing unit.

[0105] Furthermore, the image generation system also includes a first filter and a second filter, wherein the first filter is disposed between the first camera and the beam splitter, and the second filter is disposed between the second camera and the beam splitter.

[0106] The image processing unit can filter the first light through the first filter to obtain the filtered first light, and then transmit the filtered first light to the first camera.

[0107] In addition, the second light can be filtered by a second filter to obtain the filtered second light, and then the filtered second light can be transmitted to the second camera.

[0108] Furthermore, the image processing unit can perform a Fourier transform on the fitted first image to obtain the Fourier transform value of the first image, and perform a Fourier transform on the second image to obtain the Fourier transform value of the second image. Based on the Fourier transform values ​​of the first image and the second image, the Fourier transform value corresponding to the image of the target object before it is interfered with by the light propagation medium is determined. An inverse Fourier transform is then performed on the Fourier transform value corresponding to the processed image to obtain the image of the target object before it is interfered with by the light propagation medium.

[0109] In addition, the image processing unit can also input the first image and the second image into a pre-calibrated restoration model, so that the restoration model can restore the second image based on the first image to obtain the image of the target object before it is interfered with by the light propagation medium.

[0110] Furthermore, the second image can be restored based on the first image using the restoration model to obtain an image of the target object before it was interfered with by the light propagation medium.

[0111] Before being deployed to the image processing unit, the restoration model needs to be calibrated. The calibration method for the restoration model's parameters can be as follows: inputting a first image sample (set at a specified light source point on the target object) and a second image sample of the target object into the restoration model, the restoration model uses the first image sample to restore the second image sample, thus obtaining an image of the target object before it is affected by the light propagation medium. The calibration target is to minimize the deviation between the generated image of the target object before it is affected by the light propagation medium and the actual image of the target object. The parameters in the restoration model are then calibrated to obtain the calibrated restoration model.

[0112] As can be seen from the above, the natural light reflected back by the target object can be acquired through an imaging lens. A beam splitter can then be used to filter out the portion of the acquired light containing image information corresponding to a designated light source point on the target object (i.e., light in a designated wavelength band), which serves as the first ray. The remaining light is used as the second ray. A first image corresponding to the designated light source point can then be generated based on the first ray. Since the first image is the image of the designated light source point under conditions of light propagation medium interference, if the first image is not interfered with by the light propagation medium, the acquired first image should be a single light point. However, if the first image is interfered with by the light propagation medium, the acquired first image is a halo diffused from the aforementioned light point. Therefore, the degree of halo diffusion in the first image can determine the degree of influence of the light propagation medium on the acquired image of the target object. Based on this, the image of the target object affected by the interference can be restored, generating the image of the target object before the interference by the light propagation medium. This can improve the clarity of the generated image of the target object while reducing the cost of generating the image.

[0113] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 One method provided.

[0114] This instruction manual also provides Figure 5 The one shown corresponds to Figure 1A schematic diagram of the structure of an electronic device. (e.g.) Figure 5 As shown, at the hardware level, this electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above. Figure 1 The method described.

[0115] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0116] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0117] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0118] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0119] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0120] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0122] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0123] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0124] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0125] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0126] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0127] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0128] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0129] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0130] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0131] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. An image generation system, characterized in that, The image generation system includes: an optical imaging unit and an image processing unit, wherein the optical imaging unit includes: an imaging lens and a beam splitter; The imaging lens is used to acquire light from the target object and transmit the light to the beam splitter; The beam splitter is used to filter the light rays, filtering out light rays in a specified wavelength band as the first light ray, and filtering out light rays in other wavelength bands besides the first light ray as the second light ray. The first light signal corresponding to the first light ray and the second light signal corresponding to the second light ray are transmitted to the image processing unit. The first light signal is used to represent the image information of the image generation system acquired by the image generation system at a specified light source point on the target object under the condition of interference from the light propagation medium. The second light signal is used to represent the image information of the image generation system acquired by the image generation system that does not contain the light emitted by the specified light source point under the condition of interference from the light propagation medium. The image processing unit is used to obtain a first image based on the first light signal, wherein the first image is an image diffused from the designated light source point under the condition of interference from the light propagation medium; Based on the image information of the first image, the diffusion degree of the halo in the first image is determined, and based on the diffusion degree, the second image obtained based on the second light signal is restored to generate an image of the target object before it is interfered with by the light propagation medium; the diffusion degree is used to characterize the degree of influence of the light propagation medium on the collected image of the target object; the diffusion degree is determined by taking the first image of the specified light source point before it is interfered with by the light propagation medium as the standard image, and based on the difference between the first image of the specified light source point after it is interfered with by the light propagation medium and the standard image.

2. The image generation system as described in claim 1, characterized in that, The optical imaging unit further includes: a first camera and a second camera, wherein the first camera and the second camera are respectively disposed in two different directions of the beam splitter; The first camera is used to receive the first light ray transmitted by the beam splitter, generate a first light signal corresponding to the first light ray based on the first light ray, and transmit the first light signal to the image processing unit. The second camera is used to receive the second light ray transmitted by the beam splitter, generate a second light signal corresponding to the second light ray based on the second light ray, and transmit the second light signal to the image processing unit.

3. The image generation system as described in claim 2, characterized in that, The optical imaging unit further includes: a first filter and a second filter, wherein the first filter is disposed between the first camera and the beam splitter, and the second filter is disposed between the second camera and the beam splitter; The first filter is used to filter the first light to obtain filtered first light, and then transmits the filtered first light to the first camera; The second filter is used to filter the second light to obtain filtered second light, and then transmits the filtered second light to the second camera.

4. The image generation system as described in claim 1, characterized in that, The image processing unit is used to perform a Fourier transform on the first image to obtain the value of the first image after the Fourier transform. as well as Perform a Fourier transform on the second image to obtain the value of the second image after the Fourier transform; Based on the Fourier transform values ​​of the first image and the second image, determine the Fourier transform values ​​corresponding to the image of the target object before it is interfered with by the light propagation medium. Perform an inverse Fourier transform on the value corresponding to the image of the target object before it is interfered with by the light propagation medium, and obtain the image of the target object before it is interfered with by the light propagation medium.

5. The image generation system as described in claim 1, characterized in that, The image processing unit is used to input the first image and the second image into a pre-calibrated restoration model, so that the restoration model restores the second image based on the first image to obtain an image of the target object before it is interfered with by the light propagation medium.

6. The image generation system as described in claim 5, characterized in that, The image generation system further includes: a calibration unit; The calibration unit is used to input the first image sample, which is collected and set at a specified light source point on the sample target, and the second image sample of the sample target into the restoration model, so that the restoration model can restore the second image sample based on the first image sample to obtain the image of the sample target before it is interfered with by the light propagation medium. The calibration target is to minimize the deviation between the generated image of the target sample before it is affected by the light propagation medium and the actual image of the target sample. The parameters in the restoration model are then calibrated to obtain the calibrated restoration model.

7. An image generation method, characterized in that, The image generation method is applied to an image generation system, which includes an optical imaging unit and an image processing unit. The optical imaging unit includes an imaging lens and a beam splitter. The method includes: The image processing unit receives a first optical signal and a second optical signal. The first optical signal is obtained by filtering the light from the target object using the beam splitter to filter out the first light rays in a specified wavelength band. The second optical signal is obtained by converting the second light rays in other wavelength bands besides the first light rays. The first optical signal is used to represent image information of a specified light source point on the target object acquired by the image generation system when the light propagation medium interferes with the image. The second optical signal is used to represent image information of the image generation system that does not contain the light emitted by the specified light source point when the light propagation medium interferes with the image. The light from the target object is acquired through the imaging lens. Based on the first light signal, a first image is obtained, wherein the first image is the image diffused from the designated light source point under the condition of interference from the light propagation medium; Based on the image information of the first image, the diffusion degree of the halo in the first image is determined, and based on the diffusion degree, the second image obtained based on the second light signal is restored to generate an image of the target object before it is interfered with by the light propagation medium; the diffusion degree is used to characterize the degree of influence of the light propagation medium on the collected image of the target object; the diffusion degree is determined by taking the first image of the specified light source point before it is interfered with by the light propagation medium as the standard image, and based on the difference between the first image of the specified light source point after it is interfered with by the light propagation medium and the standard image.

8. The method as described in claim 7, characterized in that, The optical imaging unit further includes: a first camera and a second camera, wherein the first camera and the second camera are respectively disposed in two different directions of the beam splitter; The process of filtering out the first ray of light in a specified wavelength band and converting it to obtain a first optical signal specifically includes: The beam splitter filters out light rays in a specified wavelength range, which are then used as first light rays. The first light rays are transmitted to the first camera, which generates a first light signal corresponding to the first light rays based on the first light rays. The first light signal is then transmitted to the image processing unit. Converting second light rays of other wavelengths besides the first light ray into a second optical signal specifically includes: Light of other wavelengths besides the first light is taken as the second light and transmitted to the second camera. The second camera generates a second light signal corresponding to the second light based on the second light and transmits the second light signal to the image processing unit.

9. The method as described in claim 8, characterized in that, The optical imaging unit further includes: a first filter and a second filter, wherein the first filter is disposed between the first camera and the beam splitter, and the second filter is disposed between the second camera and the beam splitter; Transmitting the first light beam to the first camera specifically includes: The first light is filtered by the first filter to obtain the filtered first light, and the filtered first light is transmitted to the first camera. Transmitting the second light beam to the second camera specifically includes: The second light is filtered by the second filter to obtain the filtered second light, and the filtered second light is transmitted to the second camera.

10. The method as described in claim 7, characterized in that, Based on the first image, the second image obtained based on the second optical signal is restored to generate an image of the target object before it is interfered with by the light propagation medium, specifically including: Perform a Fourier transform on the first image to obtain the value of the first image after the Fourier transform; and Perform a Fourier transform on the second image to obtain the value of the second image after the Fourier transform; Based on the Fourier transform values ​​of the first image and the second image, determine the Fourier transform values ​​corresponding to the image of the target object before it is interfered with by the light propagation medium. Perform an inverse Fourier transform on the values ​​corresponding to the processed image after Fourier transform to obtain the image of the target object before it is interfered with by the light propagation medium.

11. The method as described in claim 7, characterized in that, Based on the first image, the second image obtained based on the second optical signal is restored, specifically including: The first image and the second image are input into a pre-calibrated restoration model, so that the second image is restored based on the first image by the restoration model, so as to obtain the image of the target object before it is interfered with by the light propagation medium.

12. The method as described in claim 11, characterized in that, The image generation system further includes: a calibration unit; The calibration of the restoration model specifically includes: The first image sample, which is set at a specified light source point on the target object, and the second image sample of the target object are collected and input into the restoration model. The restoration model restores the second image sample based on the first image sample to obtain the image of the target object before it is interfered with by the light propagation medium. The calibration target is to minimize the deviation between the generated image of the target sample before it is affected by the light propagation medium and the actual image of the target sample. The parameters in the restoration model are then calibrated to obtain the calibrated restoration model.

13. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 7 to 12.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 7 to 12.

Images