Shooting method, device, electronic device and medium

Abstract

The present application discloses a shooting method, device, electronic device and medium, which belongs to the field of shooting technology. The method includes: when controlling an image sensor to perform exposure, obtaining motion information of a shooting object through real-sensing pixels of the image sensor, the motion information being information of the shooting object moving within the field of view of the image sensor; determining position compensation information of the image sensor based on the motion information; controlling the movement of the image sensor based on the position compensation information so that the image sensor is stationary relative to the shooting object during the exposure process; and outputting first image data after the exposure is completed.

Description

Technical Field

[0001] The present application belongs to the field of photographing technology, and specifically relates to a photographing method, device, electronic equipment and medium. Background Art

[0002] Typically, when a user uses the electronic device to photograph a subject, the electronic device can control a rolling shutter of the electronic device to expose multiple rows of pixels of the image sensor row by row, thereby obtaining a high-resolution photographic image of the subject.

[0003] However, since the photographed object may be a moving object, when the electronic device exposes some pixels corresponding to the moving object, the exposure time is relatively long, which may cause the moving object to be deformed in the obtained photographed image.

[0004] This results in poor shooting effect of the electronic device. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a shooting method, device, electronic device and medium, which can reduce the deformation of the image area where the moving object is located when the electronic device exposes part of the pixels corresponding to the moving object, thereby improving the shooting effect of the electronic device.

[0006] In a first aspect, an embodiment of the present application provides a shooting method, which is applied to a shooting device, wherein the shooting device includes an image sensor, and the method includes: while controlling the image sensor to perform exposure, obtaining motion information of a shooting object through real-sensing pixels of the image sensor, wherein the motion information is information about the movement of the shooting object within the field of view of the image sensor; based on the motion information, determining position compensation information of the image sensor; based on the position compensation information, controlling the movement of the image sensor so that the image sensor is stationary relative to the shooting object during the exposure process; and after the exposure is completed, outputting first image data.

[0007] In a second aspect, an embodiment of the present application provides a shooting device, which includes an image sensor, and the shooting device also includes: an acquisition module, a determination module, a control module and a processing module. Among them, the acquisition module is used to obtain motion information of the shooting object through the real-sensing pixels of the image sensor while controlling the image sensor to perform exposure, and the motion information is information about the movement of the shooting object within the field of view of the image sensor. The determination module is used to determine the position compensation information of the image sensor based on the motion information obtained by the acquisition module. The control module is used to control the movement of the image sensor based on the position compensation information determined by the determination module, so that the image sensor is stationary relative to the shooting object during the exposure process. The processing module is used to output the first image data after the exposure is completed.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps of the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps of the method described in the first aspect.

[0012] In an embodiment of the present application, when controlling the image sensor of the shooting device to perform exposure, the shooting device can obtain the motion information of the shooting object (i.e., the information of the shooting object moving within the field of view of the image sensor) through the real-sensing pixels of the image sensor, and then determine the position compensation information of the image sensor based on the motion information, and then control the movement of the image sensor according to the position compensation information, so that the image sensor is stationary relative to the shooting object during the exposure process, and output the first image data after the exposure is completed. Since the shooting device can first obtain the motion information of the shooting object moving within the field of view of the image sensor, and then control the movement of the image sensor of the shooting device based on the position compensation information determined by the motion information, so that the image sensor is stationary relative to the shooting object during the exposure process. Therefore, the relative position change between the shooting object and the image sensor can be reduced, and therefore, the deformation phenomenon of the image area where the moving object is located can be reduced, and thus, the shooting effect of the shooting device can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is one of the flowcharts of the shooting method provided in the embodiment of the present application;

[0014] Figure 2 This is the second flowchart of the image capturing method according to the embodiment of the present application;

[0015] Figure 3 This is the third flow chart of the shooting method provided in the embodiment of the present application;

[0016] Figure 4is a circuit connection diagram of a real-sense pixel provided in an embodiment of the present application;

[0017] Figure 5 This is the fourth flowchart of the shooting method provided in the embodiment of the present application;

[0018] Figure 6 This is the fifth flowchart of the shooting method provided in the embodiment of the present application;

[0019] Figure 7 is a schematic diagram of the structure of a photographing device provided in an embodiment of the present application;

[0020] Figure 8 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0021] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0023] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0024] The following, in conjunction with the accompanying drawings, describes in detail the shooting method, device, electronic device and medium provided in the embodiments of the present application through specific embodiments and their application scenarios.

[0025] Figure 1 FIG. 1 is a flow chart of a shooting method provided by an embodiment of the present application. Figure 1 As shown, the shooting method provided in the embodiment of the present application may include the following steps 101 to 104.

[0026] Step 101: When a shooting device controls an image sensor to perform exposure, the shooting device obtains motion information of a shooting object through real-sensing pixels of the image sensor.

[0027] In the embodiment of the present application, a rolling shutter may be provided in the photographing device.

[0028] Optionally, in an embodiment of the present application, when a user triggers a shooting device to start a shooting application, the shooting device may display a preview interface of the shooting application, turn on the camera of the shooting device, and control a rolling shutter to expose the image sensor.

[0029] Optionally, in the embodiment of the present application, the photographed subject may be a moving person, animal, object, etc.

[0030] In the embodiment of the present application, the above-mentioned motion information is the information of the motion of the photographed object within the field of view of the image sensor.

[0031] Optionally, in an embodiment of the present application, the motion information includes at least one of the following: speed (such as the motion speed in the following embodiment), direction (such as the motion direction in the following embodiment).

[0032] In one example, the shooting device can directly collect the motion information of the shooting object through a motion sensor, wherein the motion sensor can be any one of the following: an infrared sensor, a laser sensor, a dynamic vision sensor, etc.

[0033] In another example, the photographing device may obtain at least one frame of preview image while controlling the image sensor to perform exposure, and compare the preview image to determine the motion information of the photographed object.

[0034] In another example, at least one high-sensitivity pixel (for example, a pixel provided with a white filter, or a real-sensitivity pixel in the following embodiments) may be provided in the image sensor of the shooting device, so that when the shooting device controls the image sensor for exposure, the at least two high-sensitivity pixels can detect the outline of the shooting object, and when some of the at least two high-sensitivity pixels detect the outline of the shooting object, the some high-sensitivity pixels can send the position information and corresponding time information of the some high-sensitivity pixels to the shooting device, so that the shooting device can obtain motion information based on the position information and corresponding time information of the some high-sensitivity pixels.

[0035] Step 102: The photographing device determines position compensation information of the image sensor based on the motion information.

[0036] It can be understood that the position compensation information refers to the target position to which the image sensor of the photographing device moves next.

[0037] Optionally, in an embodiment of the present application, when the motion information includes speed and direction, the shooting device can first obtain the reading time of each row of pixels of the image sensor, and then determine a target position based on the speed, direction and the reading time of each row of pixels to determine multiple target positions and obtain position compensation information.

[0038] Step 103: The photographing device controls the movement of the image sensor based on the position compensation information so that the image sensor remains stationary relative to the photographed object during the exposure process.

[0039] Optionally, in an embodiment of the present application, after the shooting device controls the image sensor to move according to the position compensation information, the shooting device may execute the above steps 101 to 103 again, that is, obtain the motion information of the shooting object again, and determine a position compensation information again based on the motion information obtained again, so that the shooting device can control the image sensor to move again according to the position compensation information.

[0040] Further optionally, the shooting device may first expose the first row of pixels of the entire photosensitive area of ​​the image sensor, and after completing the exposure of the first row of pixels, read the first row of pixels, so that when reading the first row of pixels, the shooting device can control the image sensor to move according to the determined position compensation information; next, the shooting device may obtain the motion information of the shooting object again, and based on the re-acquired motion information, re-determine the position compensation information, and then expose the second row of pixels of the entire photosensitive area of ​​the image sensor, and after completing the exposure of the second row of pixels, read the second row of pixels, so that when reading the second row of pixels, the shooting device can control the image sensor to move according to the re-determined position compensation information, and so on.

[0041] It can be understood that when different rows of pixels are exposed, the position of the image sensor relative to the object does not change.

[0042] Step 104: After the exposure is completed, the shooting device outputs the first image data.

[0043] Optionally, in an embodiment of the present application, the photographing device may read each row of pixels in the entire photosensitive area through an image sensor to obtain first image data, and the photographing device may process the first image data through an image signal processor to obtain a captured image.

[0044] After obtaining the captured image, the photographing device may display the captured image in the preview interface of the photographing application, or the photographing device may directly store the captured image in a preset storage area (eg, a storage area corresponding to the “album”).

[0045] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Figure 2 As shown, after the above step 104, the shooting method provided in the embodiment of the present application may further include the following steps 201 and 202.

[0046] Step 201: The shooting device controls the image sensor to perform exposure and outputs second image data.

[0047] Optionally, in an embodiment of the present application, the shooting device may not move the image sensor and read each row of pixels in the entire photosensitive area of ​​the image sensor to obtain second image data, so that the shooting device may process the second image data through an image signal processor to obtain a captured image.

[0048] Step 202: The photographing device generates a target image according to the first image data and the second image data.

[0049] In an embodiment of the present application, the first image data is: during the exposure process, after the image sensor moves according to the position compensation information, the image data corresponding to the first image is obtained by imaging the object through the image sensor; the second image data is: during the exposure process, when the image sensor of the shooting device does not move, the image sensor is used to image the object. The image data corresponds to the image data.

[0050] It can be understood that the first image data is the image data corresponding to the image after the deformation of the photographed object is offset, but the background object is misplaced; the second image data is the image data corresponding to the image after the deformation of the photographed object is not offset, and the background object is not misplaced.

[0051] Optionally, in an embodiment of the present application, the photographing device may fuse the first image data and the second image data to obtain target image data, and then process the target image data through an image signal processor and compress it to generate a target image.

[0052] Optionally, in an embodiment of the present application, after obtaining the target image, the shooting device can display the target image in the preview interface of the shooting application, or the shooting device can directly store the target image in a preset storage area (such as the storage area corresponding to the "album").

[0053] As can be seen, since the photographing device can fuse the first image data and the second image data, the deformation phenomenon of the moving object in the target image can be reduced.

[0054] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Figure 3 As shown, the above step 201 can be specifically implemented through the following steps 201a to 201c.

[0055] Step 201a: The photographing device obtains foreground image data corresponding to the photographed object based on the first image data.

[0056] In the embodiment of the present application, the foreground image data is image data corresponding to the photographed object.

[0057] Further optionally, in an embodiment of the present application, the photographing device can determine the image area (i.e., the foreground image area) corresponding to the photographed object from the first image data, and then obtain the foreground image data corresponding to the foreground image area from the first image data.

[0058] In an example, the photographing device may acquire at least one piece of image data, and compare the at least one piece of image data with the first image data to determine a foreground image area, thereby determining the foreground image data corresponding to the foreground image area.

[0059] Step 201b: The photographing device obtains background image data corresponding to the background based on the second image data.

[0060] In the embodiment of the present application, the above-mentioned background image data is image data corresponding to the image area other than the image area corresponding to the photographed object.

[0061] Further optionally, in an embodiment of the present application, the photographing device can first determine a first image area (the first image area is the image area corresponding to the photographed object) from the second image, and then determine the background image area based on the first image area, so that the photographing device can obtain background image data corresponding to the background image area from the second image.

[0062] It should be noted that, for the description of the photographing device obtaining background image data corresponding to the background, reference can be made to the specific description of the photographing device obtaining foreground image data corresponding to the photographed object in the above embodiment, which will not be repeated in the embodiments of the present application.

[0063] Step 201c: The photographing device fuses the foreground image data and the background image data to generate a target image.

[0064] As can be seen, since the photographing device can first obtain the foreground image data after the deformation of the moving object is offset from the first image data, and obtain the background image data of the background object that is not dislocated from the second image data, and then perform image fusion processing on the foreground image data and the background image data to generate the target image, it is possible to avoid the situation where the object is deformed in the target image.

[0065] In the shooting method provided by the embodiment of the present application, when the image sensor of the shooting device is controlled to perform exposure, the shooting device can obtain the motion information of the shooting object (i.e., the information of the shooting object moving within the field of view of the image sensor) through the real-sensing pixels of the image sensor, and then determine the position compensation information of the image sensor based on the motion information, and then control the movement of the image sensor according to the position compensation information, so that the image sensor is stationary relative to the shooting object during the exposure process, and output the first image data after the exposure is completed. Since the shooting device can first obtain the motion information of the shooting object moving within the field of view of the image sensor, and then control the movement of the image sensor based on the position compensation information determined by the motion information, so that the image sensor is stationary relative to the shooting object during the exposure process. Therefore, the relative position change between the shooting object and the image sensor can be reduced, and therefore, the deformation of the image area where the moving object is located can be reduced, so that the shooting effect of the shooting device can be improved.

[0066] In the following, an example in which at least two high-sensitivity pixels (such as real-sensing pixels) are provided in the image sensor of the shooting device is used to illustrate how the shooting device obtains motion information.

[0067] Figure 4 FIG. 2 shows a circuit connection diagram of a real-sense pixel. Figure 4 As shown, a real-sense pixel circuit may include: a first current amplification module 10 , a second current amplification module 11 , an analog-to-digital conversion module 12 , a logic judgment module 13 and a signal control module 14 .

[0068] The first current amplification module 10 includes: a pixel PD1, a first end of which is grounded, and a second end of which is connected to a first end of a first capacitor C1; an NPN transistor 15, a base of which is connected to a second end of the first capacitor C1, and a base of which is also connected to a first power supply V0 via a first resistor R1, a collector of which is connected to a first end of a second capacitor C2, and a collector of which is also connected to the first power supply V0 via a second resistor R2, and an emitter of which is grounded; a second end of the second capacitor C2 is grounded via a third resistor R3, and a second end of the second capacitor C2 is also connected to a first switch tube T1.

[0069] The analog-to-digital conversion module 12 includes: an analog-to-digital converter (ADC) 16, a first end of the analog-to-digital converter ADC16 is connected to the second end of the second capacitor C2, and the second end of the analog-to-digital converter ADC16 is also connected to the first end of the logic judgment module 13;

[0070] A first end of the signal control module 14 is connected to a second end of the logic judgment module 13 , and a second end of the signal control module 14 is connected to the first switch tube T1 ;

[0071] The second current amplification module includes: a pixel PD2, a first end of which is grounded, and a second end of which is connected to a first end of a third capacitor C3; an NPN transistor 17, a base of which is connected to a second end of the third capacitor C3, a base of which is also connected to a second power supply V1 through a fourth resistor R4, a collector of which is connected to a first end of a fourth capacitor C4, a collector of which is also connected to the second power supply V1 through a fifth resistor R5, and an emitter of which is grounded; a second end of the fourth capacitor C4 is grounded through a sixth resistor R6, and a second end of the fourth capacitor C4 is also connected to a second switch tube T2; a second end of the fourth capacitor C4 is also connected to an analog-to-digital converter ADC, and a second end of the signal control module 14 is also connected to the second switch tube T2.

[0072] Among them, the function of the first current amplification module 10 is to use the photodiode PD1 to convert the light signal into a current signal first, and amplify the current signal through the resistor R1, the resistor R2 and the transistor, and then convert it into a voltage signal 1 through the resistor R3, and output the voltage signal 1 to the analog-to-digital conversion module 12.

[0073] In the analog-to-digital conversion module 12, V ADC Vref and Vref are the working voltage and reference voltage of the analog-to-digital converter (ADC) respectively. The function of the digital-to-analog conversion module is to convert analog quantities such as voltage and current into digital quantities for subsequent processing. After the voltage signal 1 output by the first current amplification module 10 is input into the analog-to-digital conversion module 12, it is compared and transformed with the reference voltage Vref to become a photosensitive digital signal 1', and the photosensitive digital signal 1' is output to the logic judgment module 13.

[0074] In the logic judgment module 13, VH and VL are the maximum and minimum values ​​of the photosensitive digital signal of the photosensitive diode PD1 at the previous moment, that is, VH and VL indicate the voltage range of the photosensitive digital signal 2' of the photosensitive diode PD1 at the previous moment. The logic judgment module 13 can compare the photosensitive digital signal 1' with the VH and VL of the photosensitive diode PD1 at the previous moment. If the photosensitive digital signal 1' exceeds the range of VH and VL, the logic judgment module 13 can output a request signal to the signal control module 14, and the request signal is used to request the output of the voltage signal output by the first current amplification module 10, that is, the voltage signal 1.

[0075] After receiving the request signal output by the logic judgment module 13, the signal control module 14 can arbitrate when to output the voltage signal 1 and output a control signal to the multiplexing switch module. The control signal is used to control the output of the voltage signal 1 through the first path. The first path includes: a first switch tube T1, a first analog signal output module and a multiplexing switch module connected in sequence.

[0076] It can be understood that the functions of the components of the second current amplifying module 11 are similar to those of the first current amplifying module 10, and the functions of the second analog signal output module and the second switch tube T2 are similar to those of the first analog signal output module and the first switch tube T1, so they are not repeated here.

[0077] The second analog signal output module and the second switch tube T2 constitute a second path for outputting the voltage signal (analog) of the second current amplification module 11 .

[0078] It should be noted that in actual implementation, the signal control module 14 can control the simultaneous output of the voltage signal 1 of the first current amplification module 10 and the voltage signal 2 of the second current amplification module 11, and control the multiplexing switch module to process the two voltage signals (i.e., voltage signal 1 and voltage signal 2), and finally the multiplexing switch module outputs voltage signal 1, voltage signal 2, and voltage signal 1+voltage signal 2.

[0079] In this way, voltage signal 1 and voltage signal 2 can be used to determine the phase difference and perform distance or speed measurement; and voltage signal 1 + voltage signal 2 is the photosensitive signal value of the real-sensing pixel (for example, the brightness information captured by the real-sensing pixel), which is used for subsequent image processing.

[0080] In the embodiment of the present application, each of the at least two real-sensing pixels can be independent of each other, and each real-sensing pixel can sense the change in the brightness of the external environment in real time according to the pixel clock frequency, convert the change in the ambient brightness into a change in current, and then convert it into a change in a digital signal.

[0081] If the change in the digital signal of some of the at least two real-sensing pixels exceeds a preset threshold, the some real-sensing pixels will report to the system to request readout, and output a data packet with position information, brightness information, and time information of the some real-sensing pixels.

[0082] It can be understood that since the change in the position of the moving object will cause the ambient brightness to change, it will cause the digital signals corresponding to certain real-sensing pixels to change; thus, the electronic device can obtain the outline of the moving object through the digital signal changes and coordinate information corresponding to the certain real-sensing pixels, thereby determining the motion information of the moving object within the field of view of the image sensor.

[0083] Optionally, in an embodiment of the present application, at least two real-sensing pixels are distributed in the image sensor according to a predetermined density, and the at least two real-sensing pixels may be arranged in an array in the image sensor of the shooting device.

[0084] Further optionally, in the embodiment of the present application, the size and density of at least two real-sense pixels can be flexibly adjusted according to the actual application scenario, which is not limited in the embodiment of the present application.

[0085] Optionally, in an embodiment of the present application, the above coordinate information may be a coordinate position in a target coordinate system; the target coordinate system is a rectangular coordinate system with a specific pixel of the image sensor as the origin. The specific pixel may be any of the following: a pixel located at the center of the image sensor, a pixel located at an end point of the image sensor.

[0086] Optionally, in the embodiment of the present application, the above-mentioned motion information includes the target speed and target direction of the photographed object. Figure 1 ,like Figure 5 As shown, the above step 101 can be specifically implemented through the following steps 101a to 101b.

[0087] Step 101a: The shooting device determines N time information and N position information through brightness change information of real-sensing pixels in the image sensor.

[0088] In an embodiment of the present application, each of the N time information corresponds to one of the N position information, where N is a positive integer greater than 1.

[0089] Further optionally, in an embodiment of the present application, each of the above-mentioned N time information is the time information when the real-sensing pixel detects the outline of the photographed object; each of the N position information is the position information when the outline of the photographed object is detected by the corresponding time information, that is, each position information is the position information of the outline of the photographed object within the field of view of the image sensor.

[0090] Further optionally, in an embodiment of the present application, the shooting device can first determine N real-sensing pixels from the real-sensing pixels in the image sensor based on brightness change information, and then receive N time information and N position information from the N real-sensing pixels to determine N time information and N position information.

[0091] Step 101b: The photographing device determines the movement speed and the movement direction based on the N time information and the N position information.

[0092] Further optionally, in an embodiment of the present application, the shooting device may first determine the maximum time information and the minimum time information from N time information, and then determine the difference between the maximum time information and the minimum time information as the movement time of the moving object.

[0093] Further optionally, in an embodiment of the present application, the shooting device can first determine the position information corresponding to the maximum time information and the position information corresponding to the minimum time information from N time information, and then determine the distance between the position information corresponding to the maximum time information and the position information corresponding to the minimum time information as the movement distance of the moving object, so that the shooting device can determine the ratio between the movement distance of the moving object and the movement time of the moving object as the movement speed of the moving object.

[0094] Further optionally, in an embodiment of the present application, the photographing device may compare the coordinate positions of the position information corresponding to the maximum time information and the position information corresponding to the minimum time information in the target coordinate system to determine the moving direction of the moving object.

[0095] It can be seen that because real-time pixels have better signal redundancy and higher precision than conventional pixels, image sensors can capture the contours of moving objects with high precision through real-sense pixels and achieve high-precision positioning of moving objects, thereby accurately obtaining the motion information of moving objects based on the position changes of moving objects in different contour images and the image acquisition time.

[0096] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Figure 6 As shown, the above step 102 can be specifically implemented by the following step 102a, and the above step 103 can be specifically implemented by the following step 103a.

[0097] Step 102a: The photographing device determines X target positions of the image sensor according to the movement speed, the movement direction and the reading time.

[0098] In the embodiments of the present application, X is a positive integer.

[0099] In the embodiment of the present application, the above reading time is the reading time of a single row of pixels of the image sensor.

[0100] Further optionally, in the embodiment of the present application, the photographing device may first obtain the reading time, and then determine X target distances according to the reading time and X movement speeds.

[0101] Further optionally, in the embodiment of the present application, the photographing device may determine X target distances by multiplying the reading time by X movement speeds.

[0102] Further optionally, in the embodiment of the present application, the photographing device determines X target positions according to X movement directions and X target distances.

[0103] Further optionally, in the embodiment of the present application, the photographing device may determine the current position of the image sensor as the position moved X times according to X movement directions and X target distances as the X target positions.

[0104] For example, assuming that the movement speed is 0.1mm / ms, the movement direction is horizontally to the right, and the reading time of each row of pixels of the image sensor is 3ms, the shooting device can determine the product of 0.1mm / ms and 3ms as the target distance, that is, 0.3mm, and then move the current position of the image sensor horizontally to the right by 0.3mm to determine it as the target position.

[0105] Step 103a: The camera controls the image sensor to move to the X target positions in sequence.

[0106] It can be understood that the shooting device can first expose a row of pixels of the image sensor, and after the exposure of the row of pixels is completed, the image sensor of the shooting device is controlled to move to the target position within the reading time of each row of pixels of the image sensor, and then the next row of pixels of the image sensor is exposed, and so on, the image sensor is controlled to move to the X target positions in sequence. During this period, when each row of pixels of the image sensor is exposed, the relative position of the moving object and the image sensor remains unchanged.

[0107] It can be seen from this that when the shooting device exposes part of the pixels corresponding to the moving object row by row, the relative position of the moving object and the image sensor remains unchanged, thereby reducing the deformation of the image area where the moving object is located.

[0108] Of course, the moving object will also have a position change in the vertical direction relative to the shooting device.

[0109] It should be noted that the RealSense pixel not only has a good ability to capture the contour information of the moving object, but also the RealSense pixel sensor has an extremely high frequency and very good real-time performance. It also has the ability to perform phase detection and obtain phase information. For example, the phase difference can be determined by the aforementioned voltage signal 1 and voltage signal 2 to perform distance measurement and speed measurement. Specifically, the vertical distance and vertical speed of the moving object relative to the sensor can be measured. Therefore, at least two RealSense pixel sensors can provide real-time position information of the moving object in the field of view of the image sensor, as well as the distance between the moving object and the shooting device.

[0110] In an embodiment of the present application, the shooting device can obtain the speed of the moving object moving away from or approaching the shooting device in the vertical direction through at least two real-sensing pixels, and then the shooting device can control the image sensor to move to multiple target positions in sequence, thereby adjusting the distance between the lens and the image sensor to reduce the deformation of the image area where the moving object is located in the vertical direction relative to the shooting device.

[0111] The shooting method provided in the embodiment of the present application can be executed by a shooting device. In the embodiment of the present application, the shooting method is executed by a shooting device as an example to illustrate the shooting device provided in the embodiment of the present application.

[0112] Figure 7 FIG. 1 shows a possible structural diagram of a shooting device involved in an embodiment of the present application. Figure 7 As shown, the shooting device 60 may include: an acquisition module 61, a determination module 62, a control module 63 and a processing module 64. The acquisition module 61 is used to acquire the motion information of the photographed object through the real-sensing pixels of the image sensor while controlling the image sensor to perform exposure. The motion information is the information of the motion of the photographed object within the field of view of the image sensor. The determination module 62 is used to determine the position compensation information of the image sensor based on the motion information acquired by the acquisition module. The control module 63 is used to control the movement of the image sensor based on the position compensation information determined by the determination module, so that the image sensor is stationary relative to the photographed object during the exposure process. The processing module 64 is used to output the first image data after the exposure is completed.

[0113] In a possible implementation, the target motion information includes the motion speed and motion direction of the photographed object. The acquisition module 61 is specifically used to determine N time information and N position information through the brightness change information of the real-sensing pixels in the image sensor, each time information corresponds to a position information, and N is a positive integer greater than 1; and based on the N time information and the N position information, respectively determine the motion speed and the motion direction.

[0114] In a possible implementation, the determination module 62 is specifically configured to determine the X target positions of the image sensor according to the movement speed, the movement direction, and the reading time. The control module 63 is specifically configured to control the image sensor to move sequentially to the X target positions determined by the determination module 62. Wherein, X is a positive integer.

[0115] In a possible implementation, the processing module 64 is further configured to control the image sensor to perform exposure and output second image data; and to generate a target image according to the first image data and the second image data.

[0116] In one possible implementation, the processing module 64 is specifically used to obtain foreground image data corresponding to the photographed object based on the first image data; and to obtain background image data corresponding to the background based on the second image data; and to perform image fusion on the foreground image data and the background image data to generate the target image.

[0117] The shooting device provided by the embodiment of the present application can first obtain the motion information of the shooting object moving in the field of view of the image sensor, and then control the motion of the image sensor of the shooting device based on the position compensation information determined by the motion information, so that the image sensor is stationary relative to the shooting object during the exposure process. Therefore, the relative position change between the shooting object and the image sensor can be reduced, and thus the deformation phenomenon of the image area where the moving object is located can be reduced, so that the shooting effect of the shooting device can be improved.

[0118] The shooting device in the embodiment of the present application can be an electronic device or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile internet device (mobile internet device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (network attached storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0119] The shooting device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0120] The shooting device provided in the embodiment of the present application can achieve Figures 1 to 7 The various processes implemented in the method embodiment achieve the same technical effect and will not be described again here to avoid repetition.

[0121] Optionally, in the embodiment of the present application, Figure 8 As shown, an embodiment of the present application also provides an electronic device 80, including a processor 81 and a memory 82, wherein the memory 82 stores programs or instructions that can be executed on the processor 81, and when the program or instruction is executed by the processor M01, the various process steps of the above-mentioned shooting method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0122] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0123] Fig. 9 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0124] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110.

[0125] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig. 9 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0126] In the embodiment of the present application, the electronic device also includes an image sensor.

[0127] Among them, the processor 110, when controlling the image sensor for exposure, obtains motion information of the photographed object through the real-sensing pixels of the image sensor, and the motion information is the information of the motion of the photographed object within the field of view of the image sensor; based on the motion information, determines the position compensation information of the image sensor; based on the position compensation information, controls the movement of the image sensor so that the image sensor is stationary relative to the photographed object during the exposure process; after the exposure is completed, outputs the first image data.

[0128] In the shooting method provided by the embodiment of the present application, the electronic device can first obtain the motion information of the shooting object moving in the field of view of the image sensor, and then control the movement of the image sensor based on the position compensation information determined by the motion information, so that the image sensor is stationary relative to the shooting object during the exposure process. Therefore, the relative position change between the shooting object and the image sensor can be reduced, and thus the deformation phenomenon of the image area where the moving object is located can be reduced, so that the shooting effect of the electronic device can be improved.

[0129] Optionally, in an embodiment of the present application, the above-mentioned motion information includes motion speed and motion direction.

[0130] The processor 110 is specifically used to determine N time information and N position information through the brightness change information of the real-sensing pixels in the image sensor, each time information corresponds to a position information, and N is a positive integer greater than 1; and based on the N time information and the N position information, respectively determine the movement speed and the movement direction.

[0131] It can be seen that since the electronic device can obtain N time information and N position information and determine the movement speed and direction through the real-sense pixels in the image sensor, it can capture a high-precision contour map of the moving object and achieve high-precision positioning of the moving object, thereby accurately obtaining the motion information of the moving object based on the position changes of the moving object in different contour maps and the image acquisition time.

[0132] Optionally, in an embodiment of the present application, the processor 110 is specifically used to determine X target positions of the image sensor according to the movement speed, the movement direction and the reading time; and control the image sensor to move to the X target positions in sequence; wherein X is a positive integer.

[0133] It can be seen that, since the electronic device can determine X target positions according to the movement speed and the reading time, and control the image sensor to move to the X target positions in sequence, when exposing part of the pixels corresponding to the moving object row by row, the relative position of the moving object and the image sensor remains unchanged, thereby reducing the deformation of the image area where the moving object is located, and thus, the shooting effect of the electronic device can be improved.

[0134] Optionally, in the embodiment of the present application, the processor 110 is further used to control the image sensor to perform exposure and output second image data; and generate a target image according to the first image data and the second image data.

[0135] As can be seen, since the electronic device can generate a target image based on the first image data and the second image data, the deformation of the moving object in the captured image can be reduced, thus improving the shooting effect of the electronic device.

[0136] Optionally, in an embodiment of the present application, the processor 110 is specifically used to obtain foreground image data corresponding to the photographed object based on the first image data; and to obtain background image data corresponding to the background based on the second image data; and to perform image fusion on the foreground image data and the background image data to generate a target image.

[0137] It can be seen that since the electronic device can obtain foreground image data from the first image data, obtain background image data from the second image data, and fuse the foreground image data and the background image data to generate the target image, the deformation of the moving object in the captured image can be reduced, thereby improving the shooting effect of the electronic device.

[0138] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0139] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0140] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.

[0141] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned shooting method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0142] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0143] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned shooting method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0144] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0145] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned shooting method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0146] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0147] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0148] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A shooting method, applied to a shooting device, wherein the shooting device includes an image sensor, characterized in that: The method comprises: When controlling the image sensor to perform exposure, obtaining motion information of a photographed object through real-sense pixels of the image sensor, the motion information being information of the photographed object moving within the field of view of the image sensor; Based on the motion information, determining position compensation information of the image sensor; Controlling the movement of the image sensor based on the position compensation information so that the image sensor is stationary relative to the subject during exposure; After the exposure is completed, outputting the first image data; The motion information includes the motion speed and motion direction of the photographed object; The determining, based on the motion information, position compensation information of the image sensor includes: Determining X target positions of the image sensor according to the movement speed, the movement direction, and the reading time; The controlling the movement of the image sensor based on the position compensation information comprises: Controlling the image sensor to move to the X target positions in sequence; Wherein, X is a positive integer; The reading time is the reading time length of a single row of pixels of the image sensor.

2. The method according to claim 1, characterized in that The acquiring motion information of the photographed object through the real-sense pixels of the image sensor includes: Determine N time information and N position information through brightness change information of real-sense pixels in the image sensor, each time information corresponds to one position information, and N is a positive integer greater than 1; Based on the N time information and the N position information, the movement speed and the movement direction are determined respectively.

3. The method according to claim 1, characterized in that After the exposure is completed and the first image data is output, the method further includes: Controlling the image sensor to perform exposure and output second image data; A target image is generated according to the first image data and the second image data.

4. The method according to claim 3, characterized in that: Generating a target image according to the first image data and the second image data includes: Based on the first image data, obtaining foreground image data corresponding to the photographed object; Based on the second image data, obtaining background image data corresponding to the background; The foreground image data and the background image data are fused to generate the target image.

5. A photographing device, characterized in that: The photographing device includes an image sensor, and the photographing device also includes: an acquisition module, a determination module, a control module and a processing module; The acquisition module is used to acquire motion information of the photographed object through the real-sense pixels of the image sensor when controlling the image sensor to perform exposure, wherein the motion information is information about the motion of the photographed object within the field of view of the image sensor; The determining module is used to determine the position compensation information of the image sensor based on the motion information acquired by the acquiring module; The control module is used to control the movement of the image sensor based on the position compensation information determined by the determination module, so that the image sensor is stationary relative to the photographed object during the exposure process; The processing module is used to output the first image data after the exposure is completed; The motion information includes the motion speed and motion direction of the photographed object; The determination module is specifically used to determine X target positions of the image sensor according to the movement speed, the movement direction and the reading time; The control module is specifically used to control the image sensor to move to the X target positions determined by the determination module in sequence; Wherein, X is a positive integer; The reading time is the reading time length of a single row of pixels of the image sensor.

6. The photographing device according to claim 5, characterized in that: The acquisition module is specifically used to determine N time information and N position information through the brightness change information of the real-sensing pixels in the image sensor, each time information corresponds to a position information, and N is a positive integer greater than 1; and based on the N time information and the N position information, respectively determine the movement speed and the movement direction.

7. The photographing device according to claim 5, characterized in that: The processing module is further used to control the image sensor to perform exposure and output second image data; and to generate a target image according to the first image data and the second image data.

8. The photographing device according to claim 7, characterized in that: The processing module is specifically configured to obtain foreground image data corresponding to the photographed object based on the first image data; and obtain background image data corresponding to the background based on the second image data; And, the foreground image data and the background image data are fused to generate the target image.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the shooting method according to any one of claims 1 to 4 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by the processor, the steps of the shooting method according to any one of claims 1 to 4 are implemented.

Images