Signal demodulation method and device, electronic equipment and storage medium

Abstract

The invention provides a signal demodulation method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining a composite video broadcast signal and performing signal preprocessing on the composite video broadcast signal to obtain a digital sampling signal; acquiring a current filter coefficient of the filter, and performing signal filtering on the digital sampling signal by using the filter and adopting the current filter coefficient to obtain an initial brightness signal and an initial chrominance signal; determining a system of the composite video broadcast signal, and performing image restoration based on the system, the initial brightness signal and the initial chrominance signal to obtain a current image; and extracting time information and space information in the current image by using a filter adjustment network, determining a target filter coefficient adopted by the filter during next signal filtering based on the time information, the space information and the current filter coefficient, and sending the target filter coefficient to the filter by using the filter adjustment network, and during next signal filtering, the filter coefficient is dynamically adjusted, and the image definition is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic information and communication engineering, and particularly relates to a signal demodulation method, apparatus, electronic device and storage medium. Background Art

[0002] At present, a composite video broadcast signal (CVBS) is transmitted in a manner of combining luminance and chrominance. The main process of restoring a trichromatic (Red, Green, Blue) image based on the composite video broadcast signal is as follows: a signal preprocessing is performed on the composite video broadcast signal at the receiving end of the composite video broadcast signal to obtain a digital sampling signal, and then a filter is used to perform signal filtering on the digital sampling signal with fixed filter coefficients to obtain a luminance signal and a chrominance signal. Finally, an image restoration is performed based on the luminance signal and the chrominance signal to obtain an image corresponding to the composite video broadcast signal.

[0003] Due to the single filter coefficient, the frequency response ability of the filter is weak, the attenuation characteristic of the filter for high-frequency signals in the composite video broadcast signal is fixed, and the high-frequency information of the image contained in the composite video broadcast signal may be excessively attenuated, so that the chrominance signal and the luminance signal in the composite video broadcast signal cannot be accurately separated, resulting in crosstalk between the chrominance signal and the luminance signal, and further resulting in the lack of image detail information, and the clarity of the image obtained by performing image restoration based on the luminance signal and the chrominance signal is low. Therefore, how to improve the clarity of the image restored based on the composite video broadcast signal is an urgent problem to be solved at present. Summary of the Invention

[0004] The present invention provides a signal demodulation method, apparatus, electronic device and storage medium, which can solve the problem of low clarity of the image restored based on the composite video broadcast signal.

[0005] According to a first aspect of the present invention, there is provided a signal demodulation method, the method comprising:

[0006] Obtaining a composite video broadcast signal, and performing signal preprocessing on the composite video broadcast signal to obtain a digital sampling signal;

[0007] Obtaining a current filter coefficient of a filter, and using the filter to perform signal filtering on the digital sampling signal with the current filter coefficient to obtain an initial luminance signal and an initial chrominance signal;

[0008] Determining a format of the composite video broadcast signal, and performing image restoration based on the format, the initial luminance signal and the initial chrominance signal to obtain a current image;

[0009] Input the current filter coefficients and the current image into a filter adjustment network to extract temporal information and spatial information in the current image by using the filter adjustment network, determine target filter coefficients to be adopted by the filter during the next signal filtering based on the temporal information, the spatial information, and the current filter coefficients, and send the target filter coefficients to the filter by using the filter adjustment network.

[0010] According to a second aspect of the present invention, there is provided a signal demodulation device, the device comprising:

[0011] A signal preprocessing module, configured to obtain a composite video broadcast signal and perform signal preprocessing on the composite video broadcast signal to obtain a digital sampling signal;

[0012] A signal filtering module, configured to obtain current filter coefficients of a filter and perform signal filtering on the digital sampling signal by using the filter with the current filter coefficients to obtain an initial luminance signal and an initial chrominance signal;

[0013] An image restoration module, configured to determine the format of the composite video broadcast signal and perform image restoration based on the format, the initial luminance signal, and the initial chrominance signal to obtain a current image;

[0014] A coefficient determination module, configured to input the current filter coefficients and the current image into a filter adjustment network to extract temporal information and spatial information in the current image by using the filter adjustment network, determine target filter coefficients to be adopted by the filter during the next signal filtering based on the temporal information, the spatial information, and the current filter coefficients, and send the target filter coefficients to the filter by using the filter adjustment network.

[0015] According to a third aspect of the present invention, there is provided an electronic device, comprising a processor and a memory,

[0016] The memory is configured to store codes and related data;

[0017] The processor is configured to execute the codes in the memory to implement the signal demodulation method according to any one of the embodiments of the present invention.

[0018] According to a fourth aspect of the present invention, there is provided a storage medium, on which a computer program is stored, and when the program is executed by a processor, the signal demodulation method according to any one of the embodiments of the present invention is implemented.

[0019] In an embodiment of the present invention, a current composite video broadcast signal is acquired, and the current composite video broadcast signal is preprocessed to obtain a digital sampling signal; a current filter coefficient of a filter is acquired, and the digital sampling signal is filtered by using the filter with the current filter coefficient to obtain an initial luminance signal and an initial chrominance signal; an image is restored based on the initial luminance signal and the initial chrominance signal to obtain a current image; the current filter coefficient and the current image are input into a filter adjustment network, so as to extract temporal information and spatial information in the current image by using the filter adjustment network, determine a target filter coefficient for the filter to use in the next signal filtering based on the temporal information, spatial information, and the current filter coefficient, and send the target filter coefficient to the filter by using the filter adjustment network. Since an image includes temporal information and spatial information, and the temporal information and spatial information determine the frequency response ability of the filter, that is, the accuracy and distribution of the filter coefficients, and adjacent images corresponding to adjacent composite video broadcast signals have a certain correlation in temporal information and spatial information, therefore, the temporal information and spatial information in the current image can be extracted by using the filter adjustment network, and the target filter coefficient can be determined based on the temporal information, spatial information, and the current filter coefficient. Furthermore, in the next signal filtering, the current filter coefficient of the filter is dynamically adjusted to the target filter coefficient, the filter coefficient is no longer single, the frequency response ability of the filter is improved, the attenuation characteristic of the filter for high-frequency signals in the composite video broadcast signal is no longer fixed, the possibility that the high-frequency information of the image included in the composite video broadcast signal is excessively attenuated is reduced, the chrominance signal and the luminance signal in the composite video broadcast signal are more accurately separated, the crosstalk between the chrominance signal and the luminance signal is reduced, and thus the image detail information is increased, and the clarity of the image obtained by image restoration based on the composite video broadcast signal is improved. Description of the Drawings

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 It is a flowchart of a signal demodulation method provided by an embodiment of the present invention;

[0022] Figure 2 It is a schematic diagram of a ground monitoring device monitoring a composite video broadcast signal provided by an embodiment of the present invention;

[0023] Figure 3 It is a schematic diagram of a signal demodulation method provided by an embodiment of the present invention;

[0024] Figure 4 It is a schematic structural diagram of a filter adjustment network provided by an embodiment of the present invention;

[0025] Figure 5 It is a schematic diagram of a training process of a filter adjustment network provided by an embodiment of the present invention;

[0026] Figure 6 It is another schematic diagram of a signal demodulation method provided by an embodiment of the present invention;

[0027] Figure 7 It is a schematic structural diagram of a signal demodulation device provided by an embodiment of the present invention;

[0028] Figure 8 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0029] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0030] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0031] Next, the technical solutions of the present invention will be described in detail with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0032] Figure 1FIG. 0 is a schematic flowchart of a signal demodulation method provided by an embodiment of the present invention. This method can be executed by a signal demodulation device, which can be implemented in software and / or hardware. In a specific embodiment, this device can be integrated into an electronic device, such as a computer, a server, etc. The following embodiments will be described by taking the integration of this device into an electronic device as an example, referring to Figure 1 , the method can specifically include the following steps:

[0033] Step 101, obtain a composite video broadcast signal, and perform signal preprocessing on the composite video broadcast signal to obtain a digital sampling signal.

[0034] Among them, the composite video broadcast signal is an analog signal that combines multiple video information. The digital sampling signal can be understood as a digital sampling signal obtained by performing signal demodulation, signal conversion, and signal sampling on the composite video broadcast signal.

[0035] In an embodiment, performing signal preprocessing on the composite video broadcast signal to obtain a digital sampling signal may include: demodulating the composite video broadcast signal using an analog demodulator to obtain a demodulated signal, then converting the demodulated signal into a digital signal using an analog-to-digital converter, and finally sampling the digital signal to obtain a digital sampling signal.

[0036] Specifically, demodulating the composite video broadcast signal to obtain a demodulated signal may include: demodulating the composite video broadcast signal using an amplitude modulation (AM) demodulation method to obtain a demodulated signal, or demodulating the composite video broadcast signal using a frequency modulation (FM) demodulation method to obtain a demodulated signal.

[0037] Among them, amplitude modulation demodulation is the process of recovering the original low-frequency signal from the modulated amplitude modulation signal. Frequency modulation demodulation is the process of recovering the original modulation signal from the frequency modulation signal.

[0038] Exemplarily, assuming that before obtaining the composite video broadcast signal, the unmanned aerial vehicle transmits the composite video broadcast signal in an amplitude modulation manner, then the signal type of the obtained composite video broadcast signal is an amplitude modulation signal, and the amplitude modulation signal can be expressed as formula (1):

[0039] s AM (t) = (A + m(t))cos(2πf c t) (1)

[0040] Among them, s AM (t) represents the demodulated signal, A represents the amplitude modulation bias signal, m(t) represents the composite video broadcast signal, f cLet \(f_0\) be the carrier frequency and \(t\) represent time. According to formula (1), when using the amplitude modulation demodulation method to demodulate the composite video broadcast signal, the envelope detection method can be used to extract \(A + m(t)\), and after removing the offset amplitude modulation bias signal (\(A\)), the demodulated signal (\(m(t)\)) can be obtained.

[0041] Again, by way of example, assume that before acquiring the composite video broadcast signal, the drone transmits the composite video broadcast signal in a frequency modulation manner. Then, the signal type of the acquired composite video broadcast signal is a frequency modulation signal, and the frequency modulation signal can be expressed as formula (2):

[0042] s FM (t) = cos(2πf c t + ∫m(τ)dτ) (2)

[0043] Differentiating \(s FM (t)\) can obtain formula (3):

[0044]

[0045] When using the frequency modulation demodulation method to demodulate the composite video broadcast signal, a differential circuit can be first used to differentiate the composite video broadcast signal to obtain a differential signal, and then the envelope detection method can be used to process the differential signal to obtain the demodulated signal (-(2πf c + m(t))).

[0046] Since composite signals of different systems have different bandwidths. According to the low-pass sampling theorem, the sampling rate of signals of different systems needs to satisfy being greater than 2 times the highest frequency of the signal. Therefore, for common systems, when sampling digital signals, a sampling rate of 13 MHz can be selected. In other embodiments, the sampling rate of other uncommon systems can be determined according to the signal frequency.

[0047] By way of example, as Figure 2 shown, the drone captures image data during flight, encodes the image data to obtain a composite video broadcast signal, and then transmits the composite video broadcast signal to the drone controller on the ground. And the ground monitoring device needs to monitor the composite video broadcast signal transmitted by the drone to the drone controller, so as to prevent the drone from maliciously shooting and causing the infringement of ground privacy. Therefore, the ground monitoring device can acquire the composite video broadcast signal, use an analog demodulator to demodulate the composite video broadcast signal to obtain a demodulated signal, then use an analog-to-digital converter to convert the demodulated signal into a digital signal, and finally sample the digital signal to obtain a digital sampling signal.

[0048] Step 102, obtain the current filter coefficients of the filter, and use the filter to perform signal filtering on the digital sampling signal with the current filter coefficients to obtain an initial luminance signal and an initial chrominance signal.

[0049] Among them, a filter is a device or algorithm for processing luminance signals and chrominance signals in an image or video; the filter may include a luminance filter and a chrominance filter. The luminance filter is a filter for processing luminance signals in an image or video; the luminance filter may be a low-pass filter. The chrominance filter is a filter for processing chrominance signals in an image or video; the chrominance filter may be a band-pass filter. The current filter coefficients can be understood as the filter coefficients determined according to the current image corresponding to the previous signal of the composite video broadcast signal and the filter coefficients adopted by the filter during the previous signal filtering. The current filter coefficients may include luminance filter coefficients and chrominance filter coefficients. Signal filtering may include luminance signal filtering and chrominance signal filtering. The luminance signal is an electrical signal representing the brightness levels of different regions in an image. The chrominance signal is an electrical signal for representing the color information of a color image. The initial luminance signal can be understood as the signal obtained by filtering the digital sampling signal for the luminance signal by the filter using the current filter coefficients. The initial chrominance signal can be understood as the signal obtained by filtering the digital sampling signal for the chrominance signal by the filter using the current filter coefficients.

[0050] In one embodiment, using the filter to perform signal filtering on the digital sampling signal with the current filter coefficients to obtain the initial luminance signal and the initial chrominance signal may include: obtaining the luminance filter coefficients of the luminance filter, and using the luminance filter to perform luminance signal filtering on the digital sampling signal with the luminance filter coefficients to obtain the initial luminance signal; obtaining the chrominance filter coefficients of the chrominance filter, and using the chrominance filter to perform chrominance signal filtering on the digital sampling signal with the chrominance filter coefficients to obtain the initial chrominance signal.

[0051] In a specific embodiment, the luminance filter coefficients can be expressed as The chrominance filter coefficients can be expressed as

[0052] Among them, N1 represents the order of the luminance filter, and N2 represents the order of the chrominance filter. Generally, low-cost composite video broadcast signal receivers use filters of order 2-4, and the filters used by high-cost composite video broadcast signal receivers can reach up to hundreds of orders. Therefore, the value range of N1 can be 2-128. The value of N2 is usually greater than N1. h1 and h2 can be adaptively adjusted according to the digital sampling signals in different scenarios.

[0053] Exemplarily, such as Figure 3As shown, the digital sampling signal is respectively sent to the luminance filter and the chrominance filter. The luminance filter coefficients of the luminance filter are obtained, and the digital sampling signal is filtered by the luminance filter using the luminance filter coefficients to obtain the initial luminance signal; the chrominance filter coefficients of the chrominance filter are obtained, and the digital sampling signal is filtered by the chrominance filter using the chrominance filter coefficients to obtain the initial chrominance signal.

[0054] Step 103: Determine the format of the composite video broadcast signal, and perform image restoration based on the format, the initial luminance signal, and the initial chrominance signal to obtain the current image.

[0055] Among them, the format is a standard or specification that stipulates various characteristics and parameters of the signal in order to achieve effective transmission, processing, and reception of the signal. The current image can be understood as the image obtained by performing image restoration based on the information demodulated from the composite video broadcast signal.

[0056] In one embodiment, determining the format of the composite video broadcast signal may include: determining the format of the composite video broadcast signal according to the frame rate identifier of the composite video broadcast signal. In other embodiments, methods for determining the format of the composite video broadcast signal in other ways are all within the protection scope of the embodiments of the present invention.

[0057] In one embodiment, performing image restoration based on the format, the initial luminance signal, and the initial chrominance signal to obtain the current image may include: synchronizing the luminance signal of the initial luminance signal to obtain the target luminance signal; determining the target chrominance signal based on the target luminance signal and the initial chrominance signal; demodulating the target chrominance signal to obtain the color difference signal; performing image restoration based on the format, the color difference signal, and the target chrominance signal to obtain the current image.

[0058] Among them, the target luminance signal can be understood as the signal after synchronizing the luminance signal of the initial luminance signal. The target chrominance signal can be understood as the chrominance signal determined based on the target luminance signal and the initial chrominance signal. The color difference signal is a signal form that separates the luminance information and the chrominance information in the color image signal. The color difference signal may include the red color difference signal and the blue color difference signal.

[0059] Specifically, obtaining the target luminance signal by synchronizing the initial luminance signal may include: performing line synchronization on the initial luminance signal to obtain a first luminance signal; performing field synchronization on the first luminance signal to obtain a second luminance signal; differentiating the odd fields and even fields of the second luminance signal to obtain the target luminance signal. Among them, the first luminance signal can be understood as the luminance signal obtained after performing line synchronization on the initial luminance signal. The first luminance signal may include a line synchronization signal. The line synchronization signal is a periodic pulse signal, and its function is to ensure the scanning synchronization of the video image in the horizontal direction. The line synchronization signal may include a full-line synchronization signal and a half-line synchronization signal. The second luminance signal can be understood as the luminance signal obtained after performing field synchronization on the first luminance signal. The second luminance signal may include a line synchronization signal and a field synchronization signal. The field synchronization signal is a periodic pulse signal used to control the scanning synchronization of the video image in the vertical direction.

[0060] More specifically, performing line synchronization on the initial luminance signal to obtain a first luminance signal may include: obtaining the signal voltage of the initial luminance signal and performing line synchronization on the initial luminance signal according to the signal voltage to obtain the first luminance signal.

[0061] Among them, performing line synchronization on the initial luminance signal according to the signal voltage to obtain the first luminance signal may include: traversing all the initial luminance signals, and when the signal voltage belongs to a preset voltage range, determining the initial luminance signal as the line synchronization signal to obtain the first luminance signal. The preset voltage range can be understood as the voltage range composed of the signal voltages of the preset line synchronization signals. The line synchronization signal is a control signal used in the video signal to ensure the correct display and synchronization of each line of image information.

[0062] Performing field synchronization on the first luminance signal to obtain a second luminance signal may include: determining the field synchronization signal according to the line synchronization signal.

[0063] In a specific embodiment, since the field synchronization signal has the characteristics of "half line, full line, half line" before and after, the field synchronization signal can be determined according to the number of sampling points in the row where each line synchronization signal is located. Therefore, determining the field synchronization signal according to the line synchronization signal may include: determining whether the number of sampling points in the rows where three consecutive line synchronization signals are located meets a preset condition. If it meets, determining the last sampling point at the end of the line synchronization signal that meets the preset condition as the field synchronization signal. Among them, the preset condition is that the number of sampling points in each row is the first quantity, the second quantity, and the first quantity in sequence. The first quantity is the number of sampling points in the row where the half-line synchronization signal is located. The second quantity is the number of sampling points in the row where the full-line synchronization signal is located.

[0064] When the number of half-line synchronization signals between the two whole-line synchronization signals closest to the field synchronization signal is odd, the field where the field synchronization signal is located is an odd field; when the number of half-line synchronization signals between the two whole-line synchronization signals closest to the field synchronization signal is even, the field where the field synchronization signal is located is an even field. Therefore, differentiating the odd field and the even field of the second luminance signal to obtain the target luminance signal may include: determining the signal positions of the two whole-line synchronization signals closest to the field synchronization signal, determining the number of half-line synchronization signals between the two whole-line synchronization signals, if the number of half-line synchronization signals is odd, determining that the field where the field synchronization signal is located is an odd field, and if the number of half-line synchronization signals is even, determining that the field where the field synchronization signal is located is an even field.

[0065] In a composite video broadcast signal, the color synchronization signal is a segment of signal after the line synchronization signal, and the duration of the color synchronization signal after the line synchronization signal in composite video broadcast signals of different systems is different. Therefore, the color synchronization signal after the line synchronization signal can be extracted according to the duration of the color synchronization signal to improve the extraction accuracy and efficiency of the color synchronization signal. Specifically, the target luminance signal may include an odd field, an even field, and the line synchronization signals of each field. Determining the target chrominance signal based on the target luminance signal and the initial chrominance signal may include: determining the signal positions of the line synchronization signals in the order of the odd field and the even field; determining the target luminance signal within a preset duration after the signal positions as the color synchronization signal; determining the carrier frequency offset according to the color synchronization signal and the initial chrominance signal; and performing frequency synchronization on the initial chrominance signal according to the carrier frequency offset to obtain the target chrominance signal.

[0066] Among them, the preset duration can be understood as the duration of the color synchronization signal. The carrier frequency offset refers to the difference between the carrier signal frequency received at the receiving end and the carrier signal frequency transmitted at the transmitting end in a communication system. In the embodiments of the present invention, the transmitting end is a luminance filter, and the receiving end is a chrominance filter. The target chrominance signal can be understood as the signal after frequency synchronization of the chrominance signal.

[0067] In a specific embodiment, determining the signal positions of the line synchronization signals according to the odd field and the even field may include: sequentially determining the start positions of each field in the field order of the odd field and the even field, and determining the signal positions of the line synchronization signals of each field according to the start positions of each field.

[0068] Specifically, demodulating the target chrominance signal to obtain the color difference signal may include: demodulating the target chrominance signal using the quadrature coherent demodulation method to obtain the color difference signal, which can effectively separate the chrominance signal and the color difference signal and suppress noise and interference. In other embodiments, other chrominance signal demodulation methods may also be used to demodulate the target chrominance signal to obtain the color difference signal.

[0069] Specifically, image restoration is performed based on the format, color difference signals, and target chrominance signals to obtain the current image, which may include: replacing the current weight matrix with the target weight matrix according to the format; performing image restoration based on the target weight matrix, the red color difference signal, the blue color difference signal, and the target chrominance signal to obtain the current image.

[0070] Among them, the target weight matrix can be understood as the weight matrix corresponding to the format of the composite video broadcast signal.

[0071] More specifically, adjusting the current weight matrix to the target weight matrix according to the format may include: querying the matrix information according to the format to obtain the target weight matrix, and replacing the current weight matrix with the target weight matrix. In this way, the weight matrix can be automatically switched to the target weight matrix suitable for the composite video broadcast signal format according to the format, so as to achieve the purpose of adapting to the composite video broadcast signals of multiple formats.

[0072] Performing image restoration based on the target weight matrix, the red color difference signal, the blue color difference signal, and the target chrominance signal to obtain the current image may include: determining the target vector according to the red color difference signal, the blue color difference signal, and the target chrominance signal; multiplying the target weight matrix by the target vector to obtain the intermediate image data; restoring the image according to the intermediate image data to obtain the current image.

[0073] Among them, the target vector can be understood as a vector composed of the red color difference signal, the blue color difference signal, and the target chrominance signal. The intermediate image data can be understood as the pixel color value data obtained by multiplying the target weight matrix by the target vector.

[0074] Exemplarily, the target chrominance signal is Y(t), the red color difference signal is U(t), the blue color difference signal is V(t), and the target weight matrix is A. The red color difference signal, the blue color difference signal, and the target chrominance signal can be combined into a 3*1 target vector:

[0075]

[0076] Multiplying the target weight matrix by the target vector, the obtained intermediate image data is:

[0077]

[0078] Among them, is the intermediate image data, t represents time, R(t) represents the red primary color signal, G(t) represents the green primary color signal, B(t) represents the blue primary color signal, A represents a 3*3 RGB weight matrix, and · represents matrix multiplication.

[0079] In a specific embodiment, restoring an image based on intermediate image data to obtain a current image may include: obtaining the image resolution, arranging the intermediate image data according to the image resolution to restore the image and obtain the current image. The image resolution can be understood as the preset resolution of the current image. In other embodiments, other image restoration methods may also be used to process the intermediate image data to obtain the current image.

[0080] Exemplarily, as Figure 3 shown, perform luminance signal synchronization on the initial luminance signal to obtain a target luminance signal. Traverse all the initial luminance signals, and when the signal voltage belongs to a preset voltage range, determine the initial luminance signal as the line synchronization signal. Obtain the signal position of the line synchronization signal; determine the target luminance signal within a preset time period after the signal position as the color synchronization signal; determine the carrier frequency offset according to the color synchronization signal and the initial chrominance signal; perform frequency synchronization on the initial chrominance signal according to the carrier frequency offset to obtain the target chrominance signal. Demodulate the target chrominance signal using the quadrature coherent demodulation method to obtain the color difference signal. Finally, as Figure 3 shown, perform image restoration based on the video standard, the target luminance signal, and the color difference signal to obtain the current image.

[0081] Step 104: Input the current filter coefficients and the current image into the filter adjustment network, so as to use the filter adjustment network to extract the temporal information and spatial information in the current image, determine the target filter coefficients to be used by the filter in the next signal filtering based on the temporal information, spatial information, and the current filter coefficients, and use the filter adjustment network to send the target filter coefficients to the filter.

[0082] Among them, the temporal information can be understood as various information in the time dimension related to the image, and may include one or more of the image shooting time, exposure time, time stamp, temporal relationship in the image sequence, etc. In other embodiments, the temporal information may further include other information in addition to the information mentioned above. The spatial information refers to various information contained in the image in the spatial dimension, and may include one or more of the pixel position information, image size and resolution, spatial distribution of objects, geometric features of the image, spatial frequency information, etc. In other embodiments, the spatial information may further include other information in addition to the information mentioned above.

[0083] The filter adjustment network can be understood as a neural network for adjusting filter coefficients. As Figure 4 shown, the filter adjustment network may include a Convolutional Neural Network (CNN) and a Long Short-Term Memory Network (LSTM). The convolutional neural network can be used to extract the spatial information of the current image. As Figure 4As shown, the convolutional neural network may include four convolutional layers and three pooling layers. The first convolutional layer uses a 3×3 convolutional kernel with 3 channels; the second convolutional layer uses a 3×3 convolutional kernel with 8 channels; the third convolutional layer uses a 3×3 convolutional kernel with 16 channels; the fourth convolutional layer uses a 3×3 convolutional kernel with 32 channels; the pooling layer between each convolutional layer uses 4×4 pooling. A long short-term memory network is connected after the convolutional neural network. The long short-term memory network may include a flattening layer and long short-term memory units. The flattening layer can be used to convert the matrix output by the convolutional neural network into a feature vector of the current image and send it to the long short-term memory units. The long short-term memory units can be used to extract the time information of the current image and determine the target filter coefficients based on the time information, spatial information, and current filter coefficients. The long short-term memory network can output the feature vector of the current image and the filter coefficients. Figure 4 where z represents the number of steps of the long short-term memory unit. h z represents the current filter coefficients at the z-th step, which may include the luminance filter coefficient h1 and the chrominance filter coefficient h2. h z+1 represents the target filter coefficients at the (z + 1)-th step, which may include the target luminance filter coefficient and the target chrominance filter coefficient. c z represents the feature vector of the current image at the z-th step, c z+1 represents the long-term memory at the (z + 1)-th step of the long short-term memory unit. When z = 0, c z is a zero vector. The current filter coefficients also implicitly contain the temporal pattern of the image. In the long short-term memory neural unit, σ represents the sigmoid(W s ·f z +b s ) operation, where W s , b s represent the parameters of the fully connected layer, sigmoid(·) is the activation function, and f z represents the input of the sigmoid(W s ·f z +b s ) operation. Tanh represents the tanh(W z ·f z +b z ) operation, where W z , b z represent the parameters of the fully connected layer, and tanh(·) is the activation function. In the embodiments of the present invention, the filter adjustment network includes a convolutional neural network and a long short-term memory network, so that the filter adjustment network can dynamically adjust the filter coefficients (luminance filter coefficient and chrominance filter coefficient) according to the spatio-temporal relationship of different current images and the current filter coefficients to meet the image transmission requirements of different scenarios.

[0084] The target filter coefficients can be understood as the filter coefficients to be adopted by the filter during the next signal filtering, and can include the target luminance filter coefficients and the target chrominance filter coefficients.

[0085] Since there is a certain correlation between the temporal information and the spatial information of adjacent images, and the temporal information and the spatial information determine the frequency response ability of the filter, that is, the accuracy and distribution of the filter coefficients, therefore, the target filter coefficients can be determined based on the temporal information, the spatial information of the current image, and the current filter coefficients. Therefore, in one embodiment, the current filter coefficients and the current image are input into the filter adjustment network to extract the temporal information and the spatial information in the current image by using the filter adjustment network, and determine the target filter coefficients to be adopted by the filter during the next signal filtering based on the temporal information, the spatial information, and the current filter coefficients, which may include: extracting the temporal information and the spatial information in the current image by using the filter adjustment network, and determining the target luminance filter coefficients to be adopted by the luminance filter during the next luminance signal filtering and the target chrominance filter coefficients to be adopted by the chrominance filter during the next chrominance signal filtering based on the temporal information, the spatial information, the luminance filter coefficients, and the chrominance filter coefficients.

[0086] Exemplarily, as Figure 3 shown, then the current filter coefficients h z and the current image are input into the filter adjustment network to extract the temporal information and the spatial information in the current image by using the filter adjustment network, and determine the target luminance filter coefficients h1' to be adopted by the luminance filter during the next signal filtering and the target chrominance filter coefficients h2' to be adopted by the chrominance filter during the next signal filtering based on the temporal information, the spatial information, and the current filter coefficients. The filter adjustment network sends the target luminance filter coefficients h1' to the luminance filter and the target chrominance filter coefficients h2' to the chrominance filter. In this way, during the next signal filtering, the current filter coefficients of the luminance filter are dynamically adjusted to the target luminance filter coefficients, and the current filter coefficients of the chrominance filter are adjusted to the target chrominance filter coefficients. The filter coefficients are no longer single, improving the frequency response ability of the filter, so that the attenuation characteristics of the high-frequency signals in the composite video broadcast signal are no longer fixed, reducing the possibility that the high-frequency information of the images included in the composite video broadcast signal is over-attenuated, more accurately separating the chrominance signal and the luminance signal in the composite video broadcast signal, reducing the crosstalk between the chrominance signal and the luminance signal, and further increasing the image detail information and improving the image clarity of the image restored based on the luminance signal and the chrominance signal.

[0087] In an embodiment of the present invention, a filter adjustment network is used to extract temporal information and spatial information from a current image, and based on the temporal information, spatial information, and current filter coefficients, target filter coefficients are determined. Then, during the next signal filtering, the current filter coefficients of the filter are dynamically adjusted to the target filter coefficients. The filter coefficients are no longer single, improving the frequency response ability of the filter, so that the attenuation characteristics of the high-frequency signals in the composite video broadcast signal are no longer fixed, reducing the possibility that the high-frequency information of the image contained in the composite video broadcast signal is overly attenuated, more accurately separating the chrominance signal and the luminance signal in the composite video broadcast signal, reducing the crosstalk between the chrominance signal and the luminance signal, and thus increasing the image detail information and improving the image clarity of the image restored based on the composite video broadcast signal.

[0088] Figure 5 is a schematic diagram of a training process of the filter adjustment network provided by an embodiment of the present invention. As Figure 5 shown, it may specifically include the following steps:

[0089] Step 201, construct a training dataset, where the training dataset includes original images.

[0090] In one embodiment, original images and composite video broadcast signals captured by a drone in multiple scenarios can be collected, and then a training dataset can be constructed according to the original images and composite video broadcast signals.

[0091] Step 202, obtain a composite video broadcast signal, and perform signal preprocessing on the composite video broadcast signal to obtain a digital sampling signal. The composite video broadcast signal is obtained after the original image is encoded by a signal transmitting device.

[0092] In one embodiment, the composite video broadcast signal can be obtained from the training dataset, and signal preprocessing is performed on the composite video broadcast signal to obtain a digital sampling signal.

[0093] Step 203, obtain the current filter coefficients of the filter, and use the filter to perform signal filtering on the digital sampling signal with the current filter coefficients to obtain an initial luminance signal and an initial chrominance signal.

[0094] Step 204, perform image restoration based on the initial luminance signal and the initial chrominance signal to obtain the current image.

[0095] Step 205, input the current filter coefficients and the current image into the filter adjustment network to use the filter adjustment network to extract temporal information and spatial information from the current image, and determine the target filter coefficients that the filter will use during the next signal filtering based on the temporal information, spatial information, and current filter coefficients.

[0096] Step 206: Calculate the loss function value of the filter adjustment network based on the original image and the current image.

[0097] Step 207: Use the backpropagation algorithm to train the filter adjustment network according to the loss function value until a filter adjustment network that meets the training stop condition is obtained.

[0098] In the embodiment of the present invention, the filter adjustment network trained through steps 201 to 207 can extract the time information and spatial information in the current image, and determine the target filter coefficient based on the time information, spatial information, and the current filter coefficient. Furthermore, in the next signal filtering, the current filter coefficient of the filter is dynamically adjusted to the target filter coefficient, and the filter coefficient is no longer single, improving the frequency response ability of the filter, so that the attenuation characteristic of the filter for high-frequency signals in the composite video broadcast signal is no longer fixed.

[0099] The signal demodulation method provided by the embodiment of the present invention is further described below. As Figure 6 shown, Figure 6 is another flow schematic diagram of the signal demodulation method provided by the embodiment of the present invention, which may specifically include the following steps:

[0100] Step 301: Obtain a composite video broadcast signal, and perform signal preprocessing on the composite video broadcast signal to obtain a digital sampling signal.

[0101] Step 302: Obtain the current filter coefficient of the filter, and use the filter to perform signal filtering on the digital sampling signal with the current filter coefficient to obtain an initial luminance signal and an initial chrominance signal.

[0102] Step 303: Determine the format of the composite video broadcast signal.

[0103] Step 304: Perform luminance signal synchronization on the initial luminance signal to obtain a target luminance signal.

[0104] Step 305: Determine the signal position of the horizontal synchronization signal according to odd fields and even fields.

[0105] Step 306: Determine the color synchronization signal as the target luminance signal within a preset duration after the signal position.

[0106] Step 307: Determine the carrier frequency offset according to the color synchronization signal and the initial chrominance signal.

[0107] Among them, the initial chrominance signal may include the color subcarrier reference frequency. In order to achieve the compatibility of color signals and black-and-white signals and efficiently transmit chrominance information, the subcarrier modulation method is adopted. Therefore, the chrominance signal is modulated onto a color subcarrier with a specific frequency for transmission, and this frequency of the color subcarrier is the color subcarrier reference frequency.

[0108] In one embodiment, the initial chrominance signal can be multiplied by the color sync signal and then integrated to obtain an integrated value; then, the frequency of the initial chrominance signal is continuously adjusted, and the relevant operations are repeated until the integrated value reaches the maximum. At this time, the frequency of the initial chrominance signal is basically the same as the frequency of the color sync signal. The initial chrominance signal is subtracted from the color subcarrier reference frequency to obtain the carrier frequency offset.

[0109] Step 308, perform frequency synchronization on the initial chrominance signal according to the carrier frequency offset to obtain the target chrominance signal.

[0110] In one embodiment, the phase of the initial chrominance signal can be adjusted according to the carrier frequency offset to achieve frequency synchronization and obtain the target chrominance signal.

[0111] Step 309, demodulate the target chrominance signal to obtain the color difference signal.

[0112] Step 310, replace the current weight matrix with the target weight matrix according to the system format.

[0113] Step 311, determine the target vector according to the red color difference signal, the blue color difference signal, and the target chrominance signal.

[0114] Step 312, multiply the target weight matrix and the target vector to obtain the intermediate image data.

[0115] Step 313, restore the image according to the intermediate image data to obtain the current image.

[0116] Step 314, input the current filter coefficients and the current image into the filter adjustment network to extract the temporal information and spatial information in the current image by using the filter adjustment network, determine the target filter coefficients to be used by the filter in the next signal filtering based on the temporal information, spatial information, and the current filter coefficients, and send the target filter coefficients to the filter by using the filter adjustment network.

[0117] In the embodiment of the present invention, the filter adjustment network is used to extract the temporal information and spatial information in the current image, and determine the target filter coefficients based on the temporal information, spatial information, and the current filter coefficients. Furthermore, in the next signal filtering, the current filter coefficients of the filter are dynamically adjusted to the target filter coefficients. The filter coefficients are no longer single, improving the frequency response ability of the filter, making the attenuation characteristic of the filter for the high-frequency signals in the composite video broadcast signal no longer fixed, reducing the possibility that the high-frequency information of the image included in the composite video broadcast signal is excessively attenuated, more accurately separating the chrominance signal and the luminance signal in the composite video broadcast signal, reducing the crosstalk between the chrominance signal and the luminance signal, and further increasing the image detail information and improving the image clarity of the image restored based on the composite video broadcast signal.

[0118] Figure 7 This is a schematic structural diagram of a signal demodulation device provided by an embodiment of the present invention. This device is applicable to execute the signal demodulation method provided by the embodiment of the present invention. As Figure 7 shown, this device may specifically include:

[0119] A signal preprocessing module 401, configured to obtain a composite video broadcast signal and perform signal preprocessing on the composite video broadcast signal to obtain a digital sampling signal;

[0120] A signal filtering module 402, configured to obtain the current filter coefficients of a filter and use the filter to perform signal filtering on the digital sampling signal with the current filter coefficients to obtain an initial luminance signal and an initial chrominance signal;

[0121] An image restoration module 403, configured to determine the format of the composite video broadcast signal and perform image restoration based on the format, the initial luminance signal, and the initial chrominance signal to obtain a current image;

[0122] A coefficient determination module 404, configured to input the current filter coefficients and the current image into a filter adjustment network, so as to use the filter adjustment network to extract temporal information and spatial information in the current image, and determine target filter coefficients to be used by the filter in the next signal filtering based on the temporal information, the spatial information, and the current filter coefficients, and send the target filter coefficients to the filter by using the filter adjustment network.

[0123] Optionally, the image restoration module 403 performing image restoration based on the format, the initial luminance signal, and the initial chrominance signal to obtain a current image includes:

[0124] Performing luminance signal synchronization on the initial luminance signal to obtain a target luminance signal;

[0125] Determining a target chrominance signal based on the target luminance signal and the initial chrominance signal;

[0126] Demodulating the target chrominance signal to obtain a color difference signal;

[0127] Performing image restoration based on the format, the color difference signal, and the target chrominance signal to obtain the current image.

[0128] Optionally, the color difference signal includes a red color difference signal and a blue color difference signal. The image restoration module 403 performing image restoration based on the format, the color difference signal, and the target chrominance signal to obtain the current image includes:

[0129] Replace the current weight matrix with the target weight matrix according to the said format;

[0130] Perform image restoration based on the said target weight matrix, the red chrominance difference signal, the blue chrominance difference signal, and the target chrominance signal to obtain the current image.

[0131] Optionally, the image restoration module 403 performs image restoration based on the said target weight matrix, the red chrominance difference signal, the blue chrominance difference signal, and the target chrominance signal to obtain the current image, including:

[0132] Determine a target vector based on the red chrominance difference signal, the blue chrominance difference signal, and the target chrominance signal;

[0133] Multiply the said target weight matrix by the target vector to obtain intermediate image data;

[0134] Restore the image based on the intermediate image data to obtain the current image.

[0135] Optionally, the image restoration module 403 performs luminance signal synchronization on the initial luminance signal to obtain a target luminance signal, including:

[0136] Perform line synchronization on the initial luminance signal to obtain a first luminance signal;

[0137] Perform field synchronization on the first luminance signal to obtain a second luminance signal;

[0138] Distinguish the odd fields and even fields of the second luminance signal to obtain the target luminance signal.

[0139] Optionally, the target luminance signal includes odd fields, even fields, and line synchronization signals for each field. The image restoration module 403 determines the target chrominance signal based on the said target luminance signal and the initial chrominance signal, including:

[0140] Determine the signal positions of the line synchronization signals based on the odd fields and the even fields;

[0141] Determine the color synchronization signals as the target luminance signals within a preset duration after the said signal positions;

[0142] Determine the carrier frequency offset based on the color synchronization signals and the initial chrominance signal;

[0143] Perform frequency synchronization on the initial chrominance signal according to the carrier frequency offset to obtain the target chrominance signal.

[0144] Optionally, the training process of the filter adjustment network includes:

[0145] Construct a training data set, where the training data set includes original images;

[0146] Obtain a composite video broadcast signal, and perform signal preprocessing on the composite video broadcast signal to obtain a digital sampling signal, where the composite video broadcast signal is obtained after the original image is encoded by a signal transmitting device;

[0147] Obtain the current filter coefficients of the filter, and use the filter to perform signal filtering on the digital sampling signal with the current filter coefficients to obtain an initial luminance signal and an initial chrominance signal;

[0148] Perform image restoration based on the initial luminance signal and the initial chrominance signal to obtain a current image;

[0149] Input the current filter coefficients and the current image into a filter adjustment network, so as to use the filter adjustment network to extract temporal information and spatial information in the current image, and determine the target filter coefficients to be used by the filter in the next signal filtering based on the temporal information, the spatial information, and the current filter coefficients;

[0150] Calculate the loss function value of the filter adjustment network according to the original image and the current image;

[0151] Use the backpropagation algorithm to train the filter adjustment network according to the loss function value until a filter adjustment network that meets the training stop condition is obtained.

[0152] Those skilled in the art can clearly understand that for the convenience and brevity of description, only the above division of each functional module is used as an example for illustration. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the above-described functional modules can refer to the corresponding process in the foregoing method embodiments and will not be elaborated here.

[0153] The signal demodulation device provided by the embodiments of the present invention uses a filter adjustment network to extract the time information and spatial information in the current image, and determines the target filter coefficient based on the time information, spatial information, and the current filter coefficient. Then, when performing signal filtering next time, the current filter coefficient of the filter is dynamically adjusted to the target filter coefficient, and the filter coefficient is no longer single, improving the frequency response ability of the filter, so that the attenuation characteristic of the filter for the high-frequency signal in the composite video broadcast signal is no longer fixed, reducing the possibility that the high-frequency information of the image included in the composite video broadcast signal is over-attenuated, more accurately separating the chrominance signal and the luminance signal in the composite video broadcast signal, reducing the crosstalk between the chrominance signal and the luminance signal, and further increasing the image detail information, improving the clarity of the image obtained by image restoration based on the composite video broadcast signal.

[0154] Figure 8 It is a schematic diagram of the structure of an electronic device provided by the embodiments of the present invention.

[0155] Please refer to Figure 8 , an electronic device 50 is provided, including:

[0156] A processor 51; and,

[0157] A memory 52 for storing the executable instructions of the processor;

[0158] Wherein, the processor 51 is configured to execute the methods involved above by executing the executable instructions.

[0159] The processor 51 can communicate with the memory 52 through a bus 53.

[0160] The embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the methods involved above are implemented.

[0161] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disk, or optical disk that can store program codes.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A signal demodulation method, characterized in that: The method includes: Obtaining a composite video broadcast signal and performing signal preprocessing on the composite video broadcast signal to obtain a digital sampling signal; Obtaining current filter coefficients of a filter and using the filter to perform signal filtering on the digital sampling signal with the current filter coefficients to obtain an initial luminance signal and an initial chrominance signal; Determining the format of the composite video broadcast signal and performing image restoration based on the format, the initial luminance signal, and the initial chrominance signal to obtain a current image; Inputting the current filter coefficients and the current image into a filter adjustment network to use the filter adjustment network to extract temporal information and spatial information in the current image, and determining target filter coefficients for the filter to use in the next signal filtering based on the temporal information, the spatial information, and the current filter coefficients, and using the filter adjustment network to send the target filter coefficients to the filter.

2. The method according to claim 1, characterized in that The performing image restoration based on the format, the initial luminance signal, and the initial chrominance signal to obtain a current image includes: Performing luminance signal synchronization on the initial luminance signal to obtain a target luminance signal; Determining a target chrominance signal based on the target luminance signal and the initial chrominance signal; Demodulating the target chrominance signal to obtain a color difference signal; Performing image restoration based on the format, the color difference signal, and the target chrominance signal to obtain the current image.

3. The method according to claim 2, characterized in that The color difference signal includes a red color difference signal and a blue color difference signal, and the performing image restoration based on the format, the color difference signal, and the target chrominance signal to obtain the current image includes: Replacing a current weight matrix with a target weight matrix according to the format; Performing image restoration according to the target weight matrix, the red color difference signal, the blue color difference signal, and the target chrominance signal to obtain the current image.

4. The method according to claim 3, wherein The performing image restoration according to the target weight matrix, the red color difference signal, the blue color difference signal, and the target chrominance signal to obtain the current image includes: Determining a target vector according to the red color difference signal, the blue color difference signal, and the target chrominance signal; Multiplying the target weight matrix and the target vector to obtain intermediate image data; Restoring an image according to the intermediate image data to obtain the current image.

5. The method according to claim 2, characterized in that: The performing luminance signal synchronization on the initial luminance signal to obtain a target luminance signal includes: Performing line synchronization on the initial luminance signal to obtain a first luminance signal; Performing field synchronization on the first luminance signal to obtain a second luminance signal; Distinguishing the odd fields and even fields of the second luminance signal to obtain the target luminance signal.

6. The method according to claim 2, characterized in that, The target luminance signal includes odd fields, even fields, and line synchronization signals for each field. The determining a target chrominance signal based on the target luminance signal and the initial chrominance signal includes: Determining the signal positions of the line synchronization signals according to the odd fields and the even fields; Determining the color synchronization signals as the target luminance signals within a preset time period after the signal positions; Determining a carrier frequency offset according to the color synchronization signals and the initial chrominance signal; Frequency synchronization is performed on the initial chrominance signal according to the carrier frequency offset to obtain the target chrominance signal.

7. The method according to claim 1, wherein The training process of the filter adjustment network includes: Constructing a training data set, wherein the training data set includes original images; Acquire a composite video broadcast signal and perform signal preprocessing on the composite video broadcast signal to obtain a digital sampling signal, wherein the composite video broadcast signal is obtained after a signal transmitting device performs encoding processing on the original image; Obtaining a current filter coefficient of the filter, and using the filter to filter the digital sample signal using the current filter coefficient to obtain an initial luminance signal and an initial chrominance signal; Performing image restoration based on the initial luminance signal and the initial chrominance signal to obtain a current image; Inputting the current filter coefficient and the current image into a filter adjustment network, extracting temporal information and spatial information from the current image using the filter adjustment network, and determining a target filter coefficient to be used by the filter in a next signal filtering operation based on the temporal information, the spatial information, and the current filter coefficient; Calculating a loss function value of the filter adjustment network according to the original image and the current image; The filter adjustment network is trained according to the loss function value using a back propagation algorithm until a filter adjustment network that meets a training stop condition is obtained.

8. A signal demodulation device, characterized in that, The device comprises: A signal preprocessing module, configured to obtain a composite video broadcast signal and perform signal preprocessing on the composite video broadcast signal to obtain a digital sampling signal; A signal filtering module, configured to obtain a current filter coefficient of a filter, and use the filter to filter the digital sampling signal using the current filter coefficient to obtain an initial luminance signal and an initial chrominance signal; an image restoration module, configured to determine the format of the composite video broadcast signal, and perform image restoration based on the format, the initial luminance signal, and the initial chrominance signal to obtain a current image; A coefficient determination module is used to input the current filter coefficient and the current image into a filter adjustment network, so as to use the filter adjustment network to extract time information and spatial information in the current image, and determine the target filter coefficient to be adopted by the filter in the next signal filtering based on the time information, the spatial information and the current filter coefficient, and use the filter adjustment network to send the target filter coefficient to the filter.

9. An electronic device, characterized in that, Including processor and memory, The memory is used to store codes and related data; The processor is configured to execute the code in the memory to implement the signal demodulation method according to any one of claims 1 to 7.

10. A storage medium storing a computer program, wherein when the program is executed by a processor, the signal demodulation method according to any one of claims 1 to 7 is implemented.

Images