Image processing method, image processing system and electronic equipment

Abstract

The embodiment of the present application provides an image processing method, an image processing system and an electronic device. The method includes: sending a first indication message to a first image acquisition device, instructing the first image acquisition device to acquire an image including a target moving object according to a preset frame interval; sending a second indication message to a gimbal, instructing the gimbal to drive a second image acquisition device to follow the target moving object; sending a third indication message to a second image acquisition device, instructing the second image acquisition device to start acquiring a second image including the target moving object at a first target moment, and the exposure time of the first image acquisition device is less than the exposure time of the second image acquisition device; then, based on the acquired second image and the first image acquired by the first image acquisition device at the first target moment, a target image including the target moving object is generated. In this way, even if the exposure time is increased in a low-light environment, no ghosting phenomenon will occur, thereby improving the clarity of the acquired image.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing, and in particular to an image processing method, an image processing system, and an electronic device. Background Art

[0002] In modern intelligent traffic management, video technology is used to obtain various traffic data and detect and monitor illegal vehicles, which makes it easier for traffic control and management departments and road users to understand road traffic conditions in a timely manner, make accurate decisions for road traffic control and management, and choose reasonable driving routes.

[0003] However, in low-light scenarios such as rainy days, cloudy days, and nights, the camera cannot clearly capture the vehicle license plate and the people inside the vehicle, making it more difficult to detect and monitor the vehicle. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides an image processing method, an image processing system and an electronic device. In this method, after increasing the exposure time, the image acquisition device is controlled to follow the movement of the target moving object to perform image acquisition, thereby avoiding the generation of smears of the target moving object in the acquired image and improving the clarity of the acquired image.

[0005] In the first aspect, the embodiment of the present application provides an image processing method, which is applied to electronic devices such as computers, mobile phones, and tablet computers. The method includes: sending a first indication message to a first image acquisition device to instruct the first image acquisition device to acquire an image according to a preset frame interval. The image acquired by the first image acquisition device includes a target moving object, and the exposure time of the first image acquisition device is a first preset exposure time. Sending a second indication message to the gimbal to instruct the gimbal to rotate and thereby drive the second image acquisition device to follow the target moving object; and sending a third indication message to the second image acquisition device to instruct the second image acquisition device to start acquiring a second image at a first target time, the second image includes the target moving object, and the exposure time of the second image acquisition device is a second preset exposure time. Acquire the second image and the first image acquired by the first image acquisition device at the first target time; then, generate a target image based on the first image and the second image, the target image includes the target moving object; wherein the first preset exposure time is less than the second preset exposure time. In this way, for the target moving object in the first image, the brightness of the target moving object in the captured second image is increased by increasing the exposure time of the second image acquisition device when capturing the second image; and during the acquisition process, the second image acquisition device is controlled to move along with the target moving object to avoid the ghosting phenomenon of the target moving object in the captured second image; and after the target image is generated based on the first image and the second image, the clarity of the target moving object in the target image can be improved.

[0006] Exemplarily, the target moving object is a target moving vehicle.

[0007] Exemplarily, the first image acquisition device and the second image acquisition device may include: a camera, a camcorder, a still camera, and other devices with a photographing function (such as a mobile phone, etc.), and this application does not impose any limitation on this.

[0008] Exemplarily, the preset frame interval can be represented by T and can be set as required, and this application does not impose any limitation on this.

[0009] Exemplarily, the first preset exposure time can be represented by t1, and t1 can be set to be less than or equal to the short exposure time. The short exposure time can be the maximum exposure time for the first image acquisition device to shoot a moving object (the moving speed of the moving object is the preset speed) without causing a smear phenomenon. It can be set specifically as required, and the present application does not impose any restrictions on this. For example, the short exposure time threshold is 2ms, and t1 is 2ms. Exemplarily, the preset speed can be set as required, and the present application does not impose any restrictions on this. For example, the moving object is a moving vehicle, and the preset speed can be the maximum speed limit of the area where the image processing system is deployed. For another example, the preset speed can be the historical maximum speed of the area where the image processing system is deployed.

[0010] Exemplarily, the second preset exposure time can be represented by t2, and t2 can be set to be greater than or equal to the long exposure time threshold. The long exposure time threshold is greater than the minimum exposure time for the second image acquisition device to shoot a moving object (the moving speed of the moving object is the preset speed) to cause the smear phenomenon. It can be set according to the needs, and this application does not limit this. For example, if the long exposure time threshold is 20ms, t2 can be set to 30ms. Exemplarily, the long exposure time threshold is greater than the short exposure time.

[0011] Exemplarily, the second image acquisition device has no fixed frame rate.

[0012] Exemplarily, the time distance between the first target moment and the current moment is T-t1.

[0013] According to the first aspect, the method also includes: acquiring a third image, the third image is acquired by the first image acquisition device, and the third image is the previous frame image of the first image; predicting whether the target moving object in the third image enters a preset shooting trigger area at the first target moment; if it enters the preset shooting trigger area, generating a second indication information and generating a third indication information.

[0014] Exemplarily, the shooting trigger area can be a rectangular area formed by the four vertices [(Area_x0, A1), (Area_x1, A1), (Area_x0, A2), (Area_x1, A2)]. Among them, Area_x0 is the horizontal coordinate of the left line of the target area, Area_x1 is the horizontal coordinate of the right line of the target area, and the difference between Area_x0 and Area_x1 is the width of the target area; A1 and A2 are vertical coordinates, and the distance between (Area_x0, A1), (Area_x1, A1) and the bottom line of the image is equal to L1-Q1, and the distance between (Area_x0, A2), (Area_x1, A2) and the bottom line of the image is equal to L1+Q1, wherein Q1 is a number greater than or equal to 0 and less than L1, which can be set according to requirements, and the present application does not impose any restrictions on this. L1 is equal to one third of the image height.

[0015] For example, the target area is the capture area, which can be set as required, such as Figure 5a Area B shown.

[0016] For example, the present application does not limit the shape of the shooting trigger area, and the present application uses the example that the area corresponding to the shooting trigger area is a rectangle as an example for illustrative explanation.

[0017] According to the first aspect, or any implementation method of the first aspect above, second indication information is generated, including: predicting the position of the target moving object in the third image at a second target moment, wherein the second target moment is after the first target moment and is a second preset exposure time away from the first target moment; searching for preset relationship information based on the position to determine the target angle of the gimbal, the relationship information including relationship information between the position of the moving object and the angle of the gimbal; determining the target rotation speed of the gimbal based on the current angle of the gimbal, the target angle and the second preset exposure time; and generating the second indication information based on the first target moment and the target rotation speed.

[0018] Exemplarily, the current angle and target angle of the gimbal are both angles that use the initial angle of the gimbal as a reference. The initial angle of the gimbal can be calibrated in advance. For example, when the image processing system (including the first image acquisition device, the second image acquisition device and the gimbal) leaves the factory, the initial position of the gimbal can be calibrated first. Exemplarily, when the center of the image captured by the first image acquisition device and the center of the image captured by the second image acquisition device coincide with each other (or the distance between the center of the image captured by the first image acquisition device and the center of the image captured by the second image acquisition device is less than the second preset distance, and the second preset distance can be set as required, and the present application does not impose any restrictions on this), the position of the gimbal can be calibrated as the initial position of the gimbal.

[0019] Exemplarily, the target rotational velocity is a vector.

[0020] According to the first aspect, or any implementation of the first aspect above, generating third indication information includes: generating the third indication information according to the first target moment and the second preset exposure duration.

[0021] According to the first aspect, or any implementation method of the first aspect above, a target image is generated based on a second image and a first image captured by a first image acquisition device at a first target moment, including: registering the first image and the second image according to the target moving object; and fusing the area corresponding to the target moving object in the second image into the area corresponding to the target moving object in the first image.

[0022] Exemplarily, a Poisson fusion method may be used to fuse the region corresponding to the target moving object in the second image to the region corresponding to the target moving object in the first image.

[0023] Exemplarily, the position corresponding to the target moving object in the target image is the same as the position corresponding to the target moving object in the first image.

[0024] Exemplarily, the difference between the position corresponding to the target moving object in the target image and the position corresponding to the target moving object in the first image is less than a preset difference. The preset difference can be set as required, and this application does not limit this.

[0025] According to the first aspect, or any implementation of the first aspect above, the target moving object in the second image is in a preset shooting area.

[0026] Exemplarily, the preset shooting area can be set to a rectangular area formed by the four vertices [(Area_x2, A4), (Area_x3, A4), (Area_x2, A5), (Area_x3, A5)]. Among them, Area_x2 and Area_x3 are horizontal coordinates, A3 and A4 are vertical coordinates. The distance between (Area_x2, A4) and (Area_x2, A5) and the left line of the image is L2-Q2, and the distance between (Area_x3, A4) and (Area_x3, A5) and the left line of the image is L2+Q2, Q2 is less than L2, and greater than or equal to 0, and can be set specifically according to requirements, and this application does not limit this. The distance between (Area_x2, A4), (Area_x3, A4) and the bottom line of the image is equal to L1-Q1, and the distance between (Area_x2, A5), (Area_x3, A5) and the bottom line of the image is equal to L1+Q1, where Q1 is a number greater than or equal to 0 and less than L1, which can be set according to requirements and is not limited in this application. L1 is equal to one third of the image height, and L2 is equal to one half of the image width.

[0027] For example, the present application does not limit the shape of the preset shooting area, and the present application uses the example that the area corresponding to the preset shooting area is a rectangle for exemplary description.

[0028] According to the first aspect, or any implementation of the first aspect, the resolution of the target image is the same as the resolution of the first image.

[0029] According to the first aspect, or any implementation of the first aspect above, the gimbal is a two-dimensional gimbal; the rotation direction of the gimbal includes: a horizontal direction and a pitch direction.

[0030] It should be noted that the pan / tilt head of the present application is not limited to a two-dimensional pan / tilt head, and can be any type of pan / tilt head currently available, such as a ball pan / tilt head, a three-dimensional pan / tilt head, a gear pan / tilt head, a panoramic pan / tilt head, a cantilever pan / tilt head, a hydraulic pan / tilt head, a camera boom, etc., or a new type of pan / tilt head produced with the development of technology. The present application takes a two-dimensional pan / tilt head as an example for illustrative description.

[0031] According to the first aspect, or any implementation of the first aspect above, the focal length of the first image acquisition device is smaller than the focal length of the second image acquisition device. Thus, the field of view of the image acquired by the first image acquisition device is larger than the field of view of the image acquired by the second image acquisition device.

[0032] In a second aspect, an embodiment of the present application provides an image processing system, the image acquisition system comprising: a first image acquisition device, a pan / tilt head, and a second image acquisition device; wherein:

[0033] A first image acquisition device, used to acquire images according to a preset frame interval and using a first preset exposure time;

[0034] A pan / tilt platform, used for supporting the second image acquisition device and driving the second image acquisition device to rotate;

[0035] A second image acquisition device, used for acquiring images using a second preset exposure time;

[0036] Among them, the first preset exposure time of the first image acquisition device is smaller than the second preset exposure time of the second image acquisition device.

[0037] According to the second aspect, the image acquisition system also includes a control device; the control device communicates with the first image acquisition device and the second image acquisition device through wired communication or wireless communication to control the first image acquisition device and the second image acquisition device to capture images; the control device communicates with the pan-tilt head through wired communication or wireless communication to control the rotation of the pan-tilt head.

[0038] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0039] In a third aspect, an embodiment of the present application provides an image processing device, characterized in that the device includes:

[0040] The instruction module is used to send first instruction information to the first image acquisition device to instruct the first image acquisition device to acquire images according to a preset frame interval, wherein the images include a target moving object; send second instruction information to the pan / tilt head to instruct the pan / tilt head to rotate and thereby drive the second image acquisition device to follow the movement of the target moving object; and send third instruction information to the second image acquisition device to instruct the second image acquisition device to start acquiring a second image at a first target time, wherein the second image includes the target moving object;

[0041] An image acquisition module, used to acquire a second image and a first image acquired by a first image acquisition device at a first target time;

[0042] An image generation module, used to generate a target image according to the first image and the second image, wherein the target image includes a target moving object;

[0043] Among them, the first preset exposure time of the first image acquisition device is smaller than the second preset exposure time of the second image acquisition device.

[0044] According to the third aspect, an indication generation module is used to acquire a third image, which is acquired by a first image acquisition device and is the previous frame image of the first image; predict whether a target moving object in the third image enters a preset shooting trigger area at a first target moment; if it enters the preset shooting trigger area, generate second indication information and generate third indication information.

[0045] According to the third aspect, or any implementation method of the third aspect above, the indication generation module is specifically used to predict the first position of the target moving object in the third image at the second target moment, wherein the second target moment is after the first target moment and is separated from the first target moment by a second preset exposure time; searching for preset relationship information based on the first position to determine the target angle of the gimbal, the relationship information including the relationship information between the position of the moving object and the angle of the gimbal; determining the target rotation speed of the gimbal based on the current angle of the gimbal, the target angle and the second preset exposure time; and generating second indication information based on the first target moment and the target rotation speed.

[0046] According to the third aspect, or any implementation of the third aspect above, the indication generating module is specifically configured to generate third indication information according to the first target moment and the second preset exposure duration.

[0047] According to the third aspect, or any implementation method of the third aspect above, the image generation module is specifically used to align the first image and the second image according to the target moving object; and fuse the area corresponding to the target moving object in the second image into the area corresponding to the target moving object in the first image.

[0048] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0049] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, wherein the memory is coupled to the processor; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the image processing method in the first aspect or any possible implementation of the first aspect.

[0050] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0051] In a fifth aspect, an embodiment of the present application provides a chip system, comprising one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send a signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the image processing method in the first aspect or any possible implementation of the first aspect.

[0052] The fifth aspect and any implementation of the fifth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0053] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer or a processor, the computer or the processor executes the image processing method in the first aspect or any possible implementation of the first aspect.

[0054] The sixth aspect and any implementation of the sixth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the sixth aspect and any implementation of the sixth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0055] In a seventh aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when the instructions are run on a computer or a processor, enable the computer or the processor to execute the image processing method in the first aspect or any possible implementation of the first aspect.

[0056] The seventh aspect and any implementation of the seventh aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the seventh aspect and any implementation of the seventh aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1a is a schematic diagram showing an exemplary smear principle;

[0058] Figure 1b The following is a schematic diagram of an image acquisition device motion following process shown as an example;

[0059] Figure 2 is a schematic diagram of an image processing system shown as an example;

[0060] Figure 3a A structural block diagram of an image acquisition device shown as an example;

[0061] Figure 3b A structural block diagram of an image acquisition device shown as an example;

[0062] Figure 4 A schematic diagram of the information establishment process is shown as an example;

[0063] Figure 5a is a schematic diagram of a target area shown as an example;

[0064] Figure 5b is a schematic diagram of vehicle position distribution shown as an example;

[0065] Figure 5cA schematic diagram showing a shooting trigger area is shown as an example;

[0066] Figure 6 A schematic diagram of a processing process is shown as an example;

[0067] Figure 7 is a schematic diagram of an image acquisition process shown as an example;

[0068] Figure 8a is a schematic diagram of an image shown as an example;

[0069] Figure 8b is a schematic diagram of an image shown as an example;

[0070] Figure 8c is a schematic diagram of an image shown as an example;

[0071] Fig. 9 is a schematic structural diagram of an image processing device shown as an example;

[0072] Fig.10 Schematic diagram of the structure of the device shown as an example. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0074] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0075] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0076] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0077] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0078] Exemplarily, the moving object may include all moving objects, and the present application uses the moving object being a moving vehicle as an example for illustrative description.

[0079] Exemplarily, the image acquisition device may include: a camera, a camcorder, a still camera, and other devices with a photographing function (such as a mobile phone, etc.), which is not limited in this application.

[0080] Figure 1a FIG. 1 is a schematic diagram showing an exemplary embodiment of the smear principle.

[0081] Reference Figure 1a For example, the vehicle moves in a straight line. When the vehicle is at W1, the imaging position of the vehicle on the image sensor of the image acquisition device is W1'. When the vehicle moves to W2, the imaging position of the vehicle on the image sensor moves to W2'. The distance between W1 and W2 is D, and the distance between W1' and W2' is d. If the image sensor is always in an exposed state during the period when the vehicle moves from W1 to W2, the pixels between W1' and W2' can capture the light signal reflected by the moving vehicle, thereby generating a smear.

[0082] Based on the principle of ghosting, the present application considers that the image acquisition device can be controlled to follow the movement of the vehicle during the long exposure period of the image acquisition device, so that the imaging position of the moving vehicle on the image sensor remains unchanged, thereby eliminating motion blur and avoiding the ghosting phenomenon.

[0083] Figure 1b The figure is a schematic diagram showing an exemplary motion following process of an image acquisition device.

[0084] Reference Figure 1b For example, when the vehicle moves a distance D from W1 to W2 along a straight line, the image sensor can follow the movement of the vehicle, the movement distance d. In this way, when the vehicle is at W1, the imaging position of the vehicle on the image sensor is W1', and when the vehicle moves to W2, the imaging position of the vehicle on the image sensor is still W1'. Furthermore, during the period when the vehicle moves from W1 to W2, even if the image sensor is always in the exposure state, since the imaging position of the moving vehicle on the image sensor remains unchanged, the image of the moving vehicle captured will not produce a ghosting phenomenon. In this way, in low-light scenes, even if the camera exposure time is increased, there will be no ghosting phenomenon, thereby improving the clarity of the captured image.

[0085] Illustratively, the present application proposes an image processing system for controlling an image acquisition device to follow a moving object to acquire images during a long exposure period of the image acquisition device, so as to improve the clarity of images acquired in low-light scenes.

[0086] For example, the image processing system can be used to monitor and detect vehicles traveling on the road. For example, the image processing system can be deployed at the entrance and exit of the city, toll booths, inter-provincial checkpoints, inter-city checkpoints, traffic security checkpoints, etc., and this application does not limit this.

[0087] Exemplarily, the image processing system may be deployed independently of other systems, or may be integrated into other systems (such as a road traffic checkpoint system) for deployment, and this application does not impose any limitation on this.

[0088] Figure 2 Schematic diagram of an image processing system shown as an example. It should be understood that Figure 2 The illustrated image processing system 200 is merely one example of an image processing system, and the image processing system 200 may have more or fewer components than those shown in the figure, may combine two or more components, or may have a different configuration of components. Figure 2 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0089] Reference Figure 2 , exemplarily, the image processing system 200 includes: a first image acquisition device 201, a second image acquisition device 202 and a pan-tilt platform 203. Exemplarily, the first image acquisition device 201 and the pan-tilt platform 203 are arranged on the same plane A of the fixing member 204, the pan-tilt platform 203 and the first image acquisition device 201 are at a first preset distance, and the second image acquisition device 202 is arranged on the pan-tilt platform 203. The first preset distance can be set as required, and this application does not limit this.

[0090] It should be noted that the fixing member 204 may be a component independent of the image processing system 200 , or may be a component belonging to the image processing system 200 , and this application does not impose any limitation on this.

[0091] Exemplarily, the pan / tilt platform 203 may include two motors: a first motor 2031 and a second motor 2032. In one possible manner, the first motor 2031 and the second motor 2032 are arranged vertically, that is, the first rotating shaft of the first motor 2031 and the second rotating shaft of the second motor 2032 are arranged vertically. It should be noted that the first motor 2031 and the second motor 2032 may also be arranged in other ways besides being arranged vertically and in parallel, and the embodiments of the present application are not limited thereto. The present application is described by taking the first motor 2031 and the second motor 2032 as an example of being arranged vertically.

[0092] Exemplarily, the first motor 2031 can be set on the plane A of the fixing member 204, and the first rotating shaft of the first motor 2031 is perpendicular to the plane A of the fixing member 204; the second motor 2032 is set on the first motor 2031, and the second rotating shaft of the second motor 2032 is parallel to the plane A of the fixing member 204.

[0093] Exemplarily, after the second motor 2032 is started, it can drive the second image acquisition device 202 to rotate around the second rotation axis, thereby driving the second image acquisition device 202 to rotate in the pitch direction. For example, the second motor 2032 can drive the second image acquisition device 202 to rotate around the second rotation axis in the direction of Eb1, and can also drive the second image acquisition device 202 to rotate around the second rotation axis in the direction of Eb2. Exemplarily, the second rotation plane of the second image acquisition device 202 rotating around the second rotation axis is perpendicular to plane A. Exemplarily, after the first motor 2031 is started, it can drive the second motor 2032 and then drive the second image acquisition device 202 to rotate around the first rotation axis, thereby driving the second image acquisition device 202 to rotate in the horizontal direction. For example, the first motor 2032 can drive the second image acquisition device 202 to rotate around the first rotation axis in the direction of Ea1, and can also drive the second image acquisition device 202 to rotate around the first rotation axis in the direction of Ea2. Exemplarily, the first rotation plane of the second image acquisition device 202 rotating around the first rotation axis is parallel to plane A.

[0094] It should be noted that the pan / tilt platform of this application is not limited to Figure 2 The two-dimensional pan-tilt head shown can be any type of pan-tilt heads currently available, or a new type of pan-tilt head produced with the development of technology. This application takes a two-dimensional pan-tilt head as an example for illustrative description.

[0095] For example, the first motor and the second motor of the gimbal can be brushless DC motors, and the motor position and speed can be closed-loop controlled. For example, the closed-loop accuracy of the motor position can be set as required, such as 0.08 degrees, and the motor torque can also be set as required depending on the load, such as greater than 0.1Nm, which is not limited in this application.

[0096] Exemplarily, the first image acquisition device 201 and the second image acquisition device 202 are arranged in the same direction. When the image processing system is deployed at a checkpoint or intersection, the first image acquisition device 201 and the second image acquisition device 202 can be arranged toward the target area. Exemplarily, the target area can refer to the capture area, and the capture area can be set as required, and this application does not limit this.

[0097] Figure 3a and Figure 3b The figure is a structural block diagram of an image acquisition device shown as an example.

[0098] Reference Figure 3a In one possible manner, the first image acquisition device may include a first lens and a first image sensor, such as Figure 3a (1). The second image acquisition device may include a second lens and a second image sensor, such as Figure 3a (2) as shown.

[0099] Reference Figure 3b In one possible manner, the first image acquisition device may include a first lens, a first image sensor and a first image processing unit, such as Figure 3b (1). The second image acquisition device may include a second lens, a second image sensor, and a second image processing unit, such as Figure 3b (2) as shown.

[0100] It should be understood that Figure 3a and Figure 3b The image acquisition device shown is only an example of an image acquisition device, and the image acquisition device may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. In other words, the modules or components included in the first image acquisition device and the second image acquisition device can be set as required, and this application does not limit this.

[0101] Exemplarily, the first lens can be used for imaging. Optionally, the first lens can select a lens with a focal length less than the first preset focal length, and the first preset focal length can be set as required, and the present application does not limit this. For example, if the first preset focal length is 30 mm, the focal length of the first lens can be set to 25 mm.

[0102] Exemplarily, the second lens can be used for imaging. Optionally, the second lens can select a lens with a focal length greater than a second preset focal length, the second preset focal length is greater than or equal to the first preset focal length, and the second preset focal length can be set as required, and this application does not limit this. For example, if the second preset focal length is 30mm, the focal length of the second lens can be set to 35mm. For another example, if the second preset focal length is 35mm, the focal length of the second lens can be set to 50mm. In other words, the focal length of the first lens is smaller than the focal length of the second lens, so that the field of view of the first lens is larger than the field of view of the second lens.

[0103] Exemplarily, the first image sensor can be used to convert the light signal incident from the first lens into an electrical signal to obtain a RAW (unprocessed) image. Exemplarily, the resolution of the first image sensor is greater than the first preset resolution, and the target surface size (i.e., the diagonal length of the target surface) is greater than the first preset size. The first preset resolution and the first preset size can be set as required, and this application does not impose any restrictions on this. For example, the first preset resolution is 9MP (Mega Pixel), and the first preset size is 1inch. That is to say, an image sensor with a resolution greater than 9MP and a target surface greater than 1inch can be selected as the first image sensor.

[0104] Exemplarily, the second image sensor can be used to convert the light signal incident from the second lens into an electrical signal to obtain a RAW (unprocessed) image. Exemplarily, the resolution of the second image sensor is greater than the second preset resolution and less than the first preset resolution, and the pixel size is greater than the second preset size. The second preset resolution and the second preset size can be set as required, and the present application does not limit this. The second preset resolution is less than the first preset resolution. For example, the second preset resolution is 3MP, and the second preset size is 9 microns. That is to say, an image sensor with a resolution greater than 3MP and less than 9MP and a pixel size greater than 9 microns can be selected as the second image sensor.

[0105] Exemplarily, the first image processing unit and the second image processing unit can be used to perform basic processing on the RAW image (such as RAW domain processing, RGB (Red Green Blue) domain processing and YUV ("Y" represents brightness (Luminance, Luma), "U" and "V" represent chrominance, Chroma) domain processing, etc.).

[0106] Exemplarily, the image acquisition system may further include a control device ( Figure 2(not shown in the figure), the control device can communicate with the first image acquisition device and the second image acquisition device through wired communication or wireless communication to control the first image acquisition device and the second image acquisition device to acquire images; the control device communicates with the pan-tilt head through wired communication or wireless communication to control the rotation of the pan-tilt head.

[0107] In this way, the control device can drive the second image acquisition device to follow the moving vehicle by controlling the rotation of the pan-tilt head, and control the second image acquisition device to capture the image of the moving vehicle to avoid tailing of the moving vehicle and improve the clarity of the image captured in low-light scenes.

[0108] Exemplarily, the relationship information between the vehicle position and the gimbal angle can be established in advance, and then the gimbal rotation can be controlled by finding the relationship information between the vehicle position and the gimbal angle to drive the second image acquisition device to follow the moving vehicle.

[0109] Exemplarily, when the image processing system leaves the factory, the initial position of the gimbal can be calibrated first. Exemplarily, when the center of the image captured by the first image acquisition device and the center of the image captured by the second image acquisition device coincide (or the distance between the center of the image captured by the first image acquisition device and the center of the image captured by the second image acquisition device is less than the second preset distance, the second preset distance can be set as required, and the present application does not impose any restrictions on this), the position of the gimbal can be calibrated as the initial position of the gimbal (that is, the first initial angle of the first motor and the second initial angle of the second motor are calibrated). In this way, after the image processing system is deployed and the image processing system is powered on each time, the gimbal of the image processing system can be rotated to the initial position without manual alignment, simplifying user operation. And the angle of the gimbal can be determined later with reference to the initial position of the gimbal.

[0110] Exemplarily, after the image processing system is powered on, the first preset shooting parameters of the first image acquisition device can be set as required, wherein the first preset shooting parameters include: a preset frame rate (or preset frame interval) and a first preset exposure time. Exemplarily, the preset frame rate of the first image acquisition device can be set to f, and the corresponding preset frame interval of the first image acquisition device is T, T=1 / f. For example, f=25fps (Frames Per Second, the number of frames transmitted per second), then T=40ms (milliseconds). Exemplarily, the first preset exposure time can be represented by t1, t1 can be set to be less than or equal to the short exposure time, and the short exposure time can be the maximum exposure time for the first image acquisition device to shoot a moving object (the moving speed of the moving object is the preset speed) without causing a smear phenomenon. It can be set specifically as required, and this application does not limit this. For example, the short exposure time threshold is 2ms, and t1 is 2ms. Exemplarily, the preset speed can be set as required, and this application does not limit this. For example, the preset speed may be the maximum speed limit of the area where the image processing system is deployed. For another example, the preset speed may be the historical maximum speed of the area where the image processing system is deployed.

[0111] Exemplarily, after the image processing system is powered on, the second preset shooting parameters of the second image acquisition device can be set as required, wherein the second preset shooting parameters include: a second preset exposure time. Exemplarily, the second preset exposure time can be represented by t2, and t2 can be set to be greater than or equal to a long exposure time threshold, and the long exposure time threshold is greater than the minimum exposure time for the second image acquisition device to shoot a moving object (the moving speed of the moving object is a preset speed) to cause a smear phenomenon. The specific setting can be based on demand, and the present application does not impose any restrictions on this. For example, if the long exposure time threshold is 20ms, t2 can be set to 30ms. Exemplarily, the long exposure time threshold is greater than the short exposure time.

[0112] Exemplarily, the second image acquisition device does not have a fixed frame rate, but determines the timing for the second image acquisition device to start image acquisition based on the detection result of the image acquired by the first image acquisition device.

[0113] Exemplarily, the first preset shooting parameter and the second preset shooting parameter may be stored in a control device, so that the control device can control the first image acquisition device according to the first preset shooting parameter, and control the second image acquisition device according to the second preset shooting parameter.

[0114] Figure 4 The figure is a schematic diagram of an exemplary relationship information establishment process.

[0115] S401, counting vehicle positions within the target area.

[0116] Exemplarily, after the first image acquisition device and the second image acquisition device of the image processing system are set, the control device can generate indication information 1 according to the first preset shooting parameter and the first duration, and send the indication information 1 to the first image acquisition device. In this way, after receiving the indication information 1, the first image acquisition device can capture images according to the first preset shooting parameters and the first duration. Exemplarily, the first duration can be set as required, such as a preset frame interval T, and this application does not limit this.

[0117] Exemplarily, after the first image acquisition device acquires images of the first duration, the area where the vehicle is captured, i.e., the target area, can be calibrated based on the images acquired by the first image acquisition device. Exemplarily, when the images acquired by the first image acquisition device are multiple frames, one frame of the image can be selected to calibrate the target area. Exemplarily, calibrating the target area means calibrating the coordinate information of the target area corresponding to the image acquired by the first image acquisition device.

[0118] Figure 5a A schematic diagram of a target area is shown as an example.

[0119] Reference Figure 5a , exemplary, Figure 5a The image captured by the first image acquisition device includes a dual lane in the same direction, a vehicle C traveling in the lane, and a green belt beside the lane. The target area can be calibrated as required. Figure 5a As shown in B.

[0120] It should be understood that Figure 5a This is only an example of the target area, and the target area can also be calibrated at other locations; in addition, the target area can be an area of ​​other shapes, which is not limited in this application.

[0121] Exemplarily, after calibrating the target area, the control device can generate indication information 2 according to the first preset shooting parameter and the second duration, and send the indication information 2 to the first image acquisition device. In this way, after receiving the indication information 2, the first image acquisition device can capture images according to the first preset shooting parameter and the second duration. Exemplarily, the second duration is greater than the first duration, and the second duration can be set as required, such as half an hour, and this application does not limit this.

[0122] Exemplarily, after the first image acquisition device acquires the second time period, it can acquire multiple frames of images, and then perform vehicle detection on each frame of the image to determine the vehicle position of each vehicle in each frame of the image. Exemplarily, for a frame of image, the vehicle contained in the image can be detected, and then the license plate of each vehicle in the image can be detected, and then the center position of the license plate of each vehicle is used as the vehicle position of each vehicle. Then, based on the coordinate information and vehicle position of the target area, the position of the vehicle in the target area is determined. In this way, the vehicle positions in the target area can be counted by the images acquired by the first image acquisition device within the second time period.

[0123] For example, the vehicle positions whose probability information is less than the probability threshold in the target area can be filtered out, and then the relationship between the filtered vehicle positions and the gimbal angle is established for the target area. The probability threshold can be set as required, and this application does not limit this.

[0124] Exemplarily, the number of times a vehicle appears at each vehicle position in the target area within the second duration can be used as the probability information of each vehicle position in the target area. Exemplarily, the ratio of the number of times a vehicle appears at each vehicle position in the target area within the second duration to the total number of times a vehicle appears at all vehicle positions in the target area within the second duration can be used as the probability information of each vehicle position in the target area. It should be understood that the present application does not limit the manner of determining the probability information of the vehicle position in the target area.

[0125] Figure 5b The figure is a schematic diagram showing the vehicle position distribution as an example.

[0126] Reference Figure 5b , exemplary, Figure 5b A point in area B corresponds to a vehicle position, and the probability information corresponding to each vehicle position is greater than or equal to the probability threshold.

[0127] Exemplarily, when the center of the vehicle's license plate in the image is at a first preset length L1 (L1 is equal to one-third of the image height) away from the bottom line of the image, the accuracy of identifying the license plate number is the highest; therefore, when the center of the vehicle's license plate is at a distance of L1 from the bottom line of the image, the first image acquisition device can capture an image and trigger the second image acquisition device to capture an image of the vehicle. Exemplarily, a shooting trigger area can be marked based on L1 and the target area. Exemplarily, the shooting trigger area can be a rectangular area formed by the four vertices [(Area_x0, A1), (Area_x1, A1), (Area_x0, A2), (Area_x1, A2)]. Among them, Area_x0 is the horizontal coordinate of the left line of the target area, Area_x1 is the horizontal coordinate of the right line of the target area, and the difference between Area_x0 and Area_x1 is the width of the target area; A1 and A2 are the vertical coordinates, and the distance between (Area_x0, A1) and (Area_x1, A1) and the bottom line of the image is equal to L1-Q1, and the distance between (Area_x0, A2) and (Area_x1, A2) and the bottom line of the image is equal to L1+Q1, wherein Q1 is a number greater than or equal to 0 and less than L1, and can be set according to requirements, and this application does not impose any restrictions on this.

[0128] For example, the present application does not limit the shape of the shooting trigger area, and the present application uses the example that the area corresponding to the shooting trigger area is a rectangle as an example for illustrative explanation.

[0129] Figure 5c The figure is a schematic diagram showing an exemplary shooting trigger area.

[0130] Reference Figure 5c For example, the shooting trigger area is [(Area_x0, A3), (Area_x1, A3), (Area_x0, A3), (Area_x1, A3)], and the distance between (Area_x0, A3), (Area_x1, A3) and the bottom line of the image is equal to L1, that is, the shooting trigger area is the shooting trigger line D formed by the two vertices.

[0131] For example, the present application takes the establishment of the relationship information between the position of a target vehicle in the target area and the PTZ angle as an example for illustrative explanation. The target vehicle position may refer to the position in the target area that is in the shooting trigger area. For example, referring to Figure 5c The target vehicle position may refer to a position below the shooting trigger line D in the target area.

[0132] For example, when a vehicle enters the shooting trigger area, the first image acquisition device and the second image acquisition device can simultaneously acquire images; then, by comparing the images acquired by the first image acquisition device and the images acquired by the second image acquisition device, the angle of the first motor that drives the second image acquisition device to rotate and the angle of the second motor are adjusted, thereby establishing the relationship information between the target vehicle position and the gimbal angle. The following steps can be referred to;

[0133] S402: Control the first image acquisition device to acquire images according to a preset frame interval.

[0134] Exemplarily, the control device may generate indication information 3 according to the first preset shooting parameter, and then send the indication information 3 to the first image acquisition device. After receiving the indication information 3, the first image acquisition device may perform image acquisition according to the preset frame interval in the first preset shooting parameter. The exposure time of each image acquisition by the first image acquisition device is t1.

[0135] For example, the preset frame interval=40ms, t1=2ms, then the first image acquisition device can be controlled to acquire an image every 40ms, and the exposure time of each image acquisition is 2ms.

[0136] S403 , predicting, based on the current frame image IMG_A1 captured by the first image capture device, whether the vehicle in the image IMG_A1 reaches the target vehicle position after moving for a third time period.

[0137] For example, when a vehicle is driving on a road, it is usually not moving in a straight line at a constant speed, but in a short period of time, the movement of the vehicle can be regarded as a straight line motion at a constant speed. Therefore, the control device of the present application can obtain the current frame image captured by the first image acquisition device, and then predict whether the vehicle reaches the target vehicle position in the target area when the first image acquisition device captures the next frame image based on the current frame image.

[0138] For example, for the convenience of explanation, the current frame image captured by the first image capture device may be referred to as image IMG_A1. The current frame image may refer to an image captured at the moment closest to the current moment.

[0139] Exemplarily, since the first preset exposure time of the first image acquisition device is t1 and the preset frame interval is T, the time from the moment when the first image acquisition device starts to acquire the next frame of image to the current moment (that is, the moment when the first image acquisition device completes acquiring image IMG_A1) is T-t1, which can be referred to as the third time length later.

[0140] Exemplarily, the duration of the vehicle's uniform linear motion can be determined according to the preset frame interval of the first image acquisition device, and the duration of the vehicle's uniform linear motion TS = m*T, where m is a positive integer and can be set as required, and the present application does not impose any restrictions on this. For example, if T = 40ms, then m = 3 can be set, that is, the duration of the vehicle's uniform linear motion TS = 3*T = 120ms, that is, the vehicle in the three frames of images continuously acquired by the first image acquisition device is in uniform linear motion.

[0141] Exemplarily, S403 may include the following steps:

[0142] S4031, determine the current speed and current position of the vehicle in image IMG_A1.

[0143] Exemplarily, it is possible to detect whether there is a vehicle in the image IMG_A1. When there is a vehicle in the image IMG_A1, the center position of the license plate of each vehicle in the image IMG_A1 can be detected, and the center position of the license plate of the vehicle in the image IMG_A1 can be determined as the current position of the vehicle. This application is exemplified by taking a vehicle in an image as an example.

[0144] Exemplarily, the current speed of the vehicle can be determined based on the center position of the vehicle license plate in the image IMG_A1 and the center position of the vehicle license plate in the previous (m-1) frame image of the image IMG_A1. Exemplarily, when (m-1) is 1, the difference between the center position of the vehicle license plate in the image IMG_A1 and the center position of the vehicle license plate in the previous frame image of the image IMG_A1 can be calculated, and then the current speed of the vehicle can be calculated based on the difference and the preset frame interval. For example, the following formula can be used:

[0145]

[0146] Among them, V is the current speed of the vehicle, (xa, ya) is the center position of the vehicle license plate in the image IMG_A1, and (xb, yb) is the center position of the vehicle license plate in the image captured in the previous frame.

[0147] Exemplarily, when (m-1) is greater than 1, the vehicle speed can be calculated based on the center position of the vehicle license plate in two adjacent frames of images to obtain (m-1) vehicle speeds, and then the average value of the (m-1) vehicle speeds is used as the current speed of the vehicle.

[0148] S4032: predicting the position P1 to be reached by the vehicle after a third period of time, based on the current speed, current position and preset frame interval of the vehicle in the image IMG_A1.

[0149] For example, assuming that the current position is (xa, ya), the current speed of the vehicle is V, the preset frame interval is T, and the first preset exposure time is t1, the position reached by the vehicle after the preset frame interval is P1 (x1, y1), where:

[0150] x1=xa

[0151] y1=ya+V*(T-t1)

[0152] S4033, determining whether the position P1 reached by the vehicle after the third period of time is the target vehicle position.

[0153] For example, assuming that the position P1 reached by the vehicle after the third time period is (x1, y1), and the position of the target vehicle is (xo, yo), then the distance LX between the position P1 reached by the vehicle after the third time period and the position of the target vehicle can be calculated using the following formula:

[0154]

[0155] Exemplarily, if the distance LX between the position P1 reached by the vehicle in the image IMG_A1 after the third time period and the position of the target vehicle is less than or equal to the third preset distance, it can be determined that the vehicle in the image IMG_A1 has reached the position of the target vehicle after the third time period. If the distance LX between the position P1 reached by the vehicle in the image IMG_A1 after the third time period and the position of the target vehicle is greater than the third preset distance, it can be determined that the vehicle in the image IMG_A1 has not reached the position of the target vehicle after the third time period. The third preset distance can be set as required, and this application does not limit this.

[0156] Exemplarily, when it is determined that the vehicle in the image IMG_A1 reaches the target vehicle position in the target area after moving for the third time period, S404 is executed; otherwise, after the first image acquisition device acquires the next frame of image, S403 is executed again.

[0157] S404, when the first image acquisition device acquires the next frame image IMG_A2, the second image acquisition device is controlled to start image acquisition to obtain image IMG_B1.

[0158] Exemplarily, when it is determined that the vehicle in the image IMG_A1 reaches the target vehicle position in the target area after the third duration of movement, the control device can generate indication information 4 according to the current time, the third duration and the target exposure duration. Exemplarily, the trigger shooting time (that is, the time when the first image acquisition device starts to capture the next frame of the image) can be determined according to the current time and the third duration; then the trigger shooting time and the target exposure duration are used to generate indication information 4, and the indication information 4 is sent to the second image acquisition device. Among them, the target exposure duration can be represented by t3, t3 can be set to be less than or equal to the short exposure duration threshold, and t3 can be equal to t1.

[0159] Exemplarily, after receiving the indication information 4, the second image acquisition device can open the shutter of the second image acquisition device after reaching the trigger shooting moment, and control the shutter of the second image acquisition device to close after the second image acquisition device is exposed for a time period of t3. Exemplarily, for the convenience of explanation, the image acquired by the second image acquisition device can be referred to as image IMG_B1.

[0160] Exemplarily, the first image acquisition device also acquires an image at the time of triggering the shooting, that is, the next frame of image IMG_A1, which can be called image IMG_A2.

[0161] Exemplarily, the image IMG_A2 and the image IMG_B1 may be compared to establish relationship information between the target vehicle position and the gimbal angle.

[0162] Exemplarily, it can be determined whether the relationship information establishment condition is satisfied based on the image IMG_A2 and the image IMG_B1. Exemplarily, the relationship information establishment condition can include: the image IMG_A2 and the image IMG_B1 include the same license plate, and the center position of the vehicle license plate in the image IMG_B1 is in a preset shooting area.

[0163] S405 , determining whether the image IMG_A2 and the image IMG_B1 contain the same license plate number.

[0164] Exemplarily, the control device may acquire image IMG_A2 and image IMG_B1, and then perform license plate recognition on image IMG_A2 to identify the license plate number contained in image IMG_A2. Also, the control device may perform license plate recognition on image IMG_B1 to identify the license plate number contained in image IMG_B1. Then, based on the license plate recognition result, it is determined whether image IMG_A2 and image IMG_B1 contain the same license plate number.

[0165] When the image IMG_A2 and the image IMG_B1 contain the same license plate number, it means that the second image acquisition device has captured an image of a vehicle with a recognizable license plate number, and S406 can be executed at this time. When the image IMG_A2 and the image IMG_B1 do not contain the same license plate number, it means that the second image acquisition device has not captured an image of a vehicle with a recognizable license plate number, and S410 can be executed to adjust the angle of the second image acquisition device.

[0166] S406, determining whether the license plate position in the image IMG_B1 is within a preset shooting area.

[0167] Exemplarily, when the center of the license plate of the vehicle in the image is at a distance of L1 from the bottom line of the image, and at a distance of a second preset length L2 (L2 is equal to half of the width of the image) from the left or right line of the image, the accuracy of identifying the license plate number is the highest. Therefore, the preset shooting area can be set to a rectangular area formed by the four vertices [(Area_x2, A4), (Area_x3, A4), (Area_x2, A5), (Area_x3, A5)]. Among them, Area_x2 and Area_x3 are horizontal coordinates, A3 and A4 are vertical coordinates. The distance between (Area_x2, A4) and (Area_x2, A5) and the left line of the image is L2-Q2, and the distance between (Area_x3, A4) and (Area_x3, A5) and the left line of the image is L2+Q2, Q2 is less than L2, and greater than or equal to 0, and can be set specifically according to requirements, and this application does not limit this. The distance between (Area_x2, A4) and (Area_x3, A4) and the bottom line of the image is equal to L1-Q1, and the distance between (Area_x2, A5) and (Area_x3, A5) and the bottom line of the image is equal to L1+Q1, where Q1 is a number greater than or equal to 0 and less than L1. It can be set according to requirements and this application does not impose any restrictions on this.

[0168] It should be noted that this application does not show the shape of the preset shooting area. This application takes the corresponding area of ​​the preset shooting area as a rectangle as an example for illustrative explanation.

[0169] For example, if the license plate position in the image IMG_B1 is in the preset shooting area, S407 can be executed. If the license plate position in the image IMG_B1 is not in the preset shooting area, S408 can be executed to adjust the angle of the second image acquisition device.

[0170] S407, establishing relationship information between the target vehicle position (X, Y) and the angle a of the first motor and the angle b of the second motor when the second image acquisition device acquires the image IMG_B1.

[0171] Exemplarily, when the license plate position in the image IMG_B1 is in the preset shooting area, the angle a of the first motor and the angle b of the second motor can be determined. Exemplarily, the current angle of the first motor and the current angle of the second motor can be determined, and the difference between the current angle of the first motor and the first initial angle of the first motor can be determined as the angle a of the first motor; and the difference between the current angle of the second motor and the second initial angle of the second motor can be determined as the angle b of the second motor.

[0172] Then, the image processing unit can establish the relationship information between (X, Y) and (a, b), so that the relationship information between the vehicle position and the pan / tilt angle can be established.

[0173] S408 , determining a first rotation direction of the first motor and a second rotation direction of the second motor according to a positional relationship between the license plate position in the image IMG_B1 and the preset shooting area.

[0174] Exemplarily, when the license plate position of the vehicle in the second image is not in the preset shooting area, the pitch angle and / or horizontal angle of the gimbal may be adjusted.

[0175] For example, if the license plate position in the image IMG_B1 is below the lower edge of the preset shooting area, it means that the upward angle of the second motor is too large. At this time, the second rotation direction of the second motor can be determined to be Figure 2 If the license plate position in image IMG_B1 and image IMG_B1 are located above the upper edge of the preset shooting area, it means that the upward angle of the second motor is too small, then the second rotation direction of the second motor can be determined to be Figure 2 That is, the pitch rotation direction of the second motor is determined.

[0176] For example, if the license plate position in the image IMG_B1 is located on the right side of the right line of the preset shooting area, it means that the first motor is Figure 2 The deflection angle in the Ea1 direction is too large. At this time, it can be determined that the first rotation direction of the first motor is Figure 2 If the license plate position in the image IMG_B1 is located to the left of the left line of the preset shooting area, it means that the first motor is rotating in the direction of Ea2. Figure 2 The deflection angle in the Ea2 direction is too large. At this time, it can be determined that the first rotation direction of the first motor is Figure 2 That is, the horizontal rotation direction of the first motor is determined.

[0177] S409, controlling the first motor to rotate in a first rotation direction by a first preset step length, and controlling the second motor to rotate in a second rotation direction by a second preset step length.

[0178] Exemplarily, the first preset step length and the second preset step length can be pre-set as needed, wherein the first preset step length and the second preset step length can be the same, such as the first preset step length and the second preset step length are both 3°. Of course, the first preset step length and the second preset step length can also be different, such as the first preset step length is 3° and the second preset step length is 2°, and this application does not limit this.

[0179] Exemplarily, the control device generates instruction information 5 according to the first rotation direction, the first preset step length, the second rotation method and the second preset step length; and then sends the instruction information 5 to the gimbal. After receiving the instruction information 5, the gimbal can control the first motor to rotate in the first rotation direction by the first preset step length to adjust the horizontal angle of the first motor; and control the second motor to rotate in the second rotation direction by the second preset step length to adjust the pitch angle of the second motor. In addition, the second image acquisition device can be driven to rotate in the first rotation direction by the first preset step length, and in the second rotation direction by the second preset step length.

[0180] Exemplarily, after completing S409 of controlling the first motor to rotate in the first rotation direction by a first preset step length, and controlling the second motor to rotate in the second rotation direction by a second preset step length, the execution may return to step S403 .

[0181] S410, controlling the second motor to rotate in a preset direction by a third preset step length.

[0182] For example, when the image IMG_A2 and the image IMG_B1 do not contain the same license plate number, the field of view of the second image acquisition device can be moved toward the lower edge of the image IMG_B1. For example, a preset direction can be set in advance for the second motor, and the preset direction can be Figure 2 The direction of rotation toward Eb2. A third preset step size can be set for the second motor, and the third preset step size can be set as required, such as 3°, and the present application does not limit this. Exemplarily, the third preset step size and the second preset step size can be the same or different, and the present application does not limit this.

[0183] Exemplarily, when the image IMG_A2 and the image IMG_B1 do not contain the same license plate number, the control device can generate indication information 6 according to the third preset step length and the preset direction; and then send the indication information 6 to the gimbal. After receiving the indication information 6, the gimbal can control the second motor to rotate in the preset direction by the third preset step length to adjust the pitch angle of the second motor, thereby driving the second image acquisition device to rotate in the preset direction by the third preset step length, so as to control the field of view of the second image acquisition device to move toward the lower edge of the image IMG_B1.

[0184] Exemplarily, after completing step S410 of controlling the first motor to rotate in a preset direction by a third preset step length, the process may return to step S403 .

[0185] For example, the relationship information between the target vehicle position and the PTZ angle can be established according to the above S401 to S410. Please refer to Table 1:

[0186] Table 1

[0187] Vehicle location The angle of the first motor The angle of the second motor (X1,Y1) a1 b1 (X2,Y2) a2 b2 (X3,Y3) a3 b3 ...... ...... ...... (Xk,Yk) ak bk

[0188] Wherein, k is a positive integer, and k represents the number of pieces of relationship information. The angle of the first motor in Table 1 refers to the angle with the first initial angle as a reference, which can be a positive number or a negative number. The angle of the second motor in Table 1 refers to the angle with the second initial angle as a reference, which can be a positive number or a negative number.

[0189] Based on the above image processing system and the established relationship information, the present application proposes an image processing method, which can be executed by a control device.

[0190] Figure 6 A schematic diagram of the processing process is shown as an example.

[0191] S601: Send first instruction information to a first image acquisition device to instruct the first image acquisition device to acquire an image according to a preset frame interval, where the image includes a target moving object.

[0192] Exemplarily, the first indication information may be generated according to the first preset shooting parameters; wherein the first indication information is used to control the first image acquisition device to acquire images according to the first preset shooting parameters.

[0193] Exemplarily, the first indication information may be generated after the image processing system is powered on; or the first indication information may be generated after receiving a monitoring operation from the user and in response to the user's operation behavior, and this application does not impose any restrictions on this.

[0194] Exemplarily, after the first indication information is generated, the first indication information may be sent to the first image acquisition device. After receiving the first indication information, the first image acquisition device acquires an image according to a preset frame interval and using a first preset exposure time. The preset frame interval is T, and the first preset exposure time is t1.

[0195] Exemplarily, the target moving object is a target moving vehicle, wherein the image captured by the first image acquisition device may include multiple moving vehicles, and any one of the moving vehicles may be determined as the target moving vehicle.

[0196] S602, sending a second indication message to the gimbal to instruct the gimbal to rotate and thereby drive the second image acquisition device to follow the movement of the target moving object; and sending a third indication message to the second image acquisition device to instruct the second image acquisition device to start acquiring a second image at the first target moment, the second image including the target moving object.

[0197] Exemplarily, a preset trigger condition for triggering the second image acquisition device to start image acquisition can be pre-set, and the preset trigger condition includes: the vehicle enters the preset shooting trigger area after moving for a third time. Then, after the first image acquisition device captures each current frame image, it is determined whether the preset trigger condition for triggering the second image acquisition device to start acquisition is met based on the current frame image. When the preset trigger condition for triggering the second image acquisition device to start acquisition is met, a second indication information and a third indication information can be generated. Then, the second indication information is sent to the gimbal to instruct the gimbal to rotate and drive the second image acquisition device to follow the target moving vehicle; and the third indication information is sent to the second image acquisition device to instruct the second image acquisition device to start capturing the second image at the first target moment. Exemplarily, the first target moment can be a moment after the current moment and a third time away from the current moment.

[0198] Exemplarily, based on the current frame image, it is determined whether the preset trigger conditions for triggering the second image acquisition device to start acquisition are met, that is, it is determined whether the target moving vehicle in the current frame image enters the preset shooting trigger area when the first image acquisition device captures the next frame.

[0199] Exemplarily, determining whether to generate the second indication information and the third indication information may include the following steps S11 to S13:

[0200] S11, acquiring a third image.

[0201] Exemplarily, for ease of description, during the application of the image processing system, the current frame image captured by the first image acquisition device is referred to as the third image.

[0202] S12, predicting whether the target moving object in the third image enters a preset shooting triggering area at the first target moment.

[0203] Exemplarily, predicting whether the target moving vehicle in the third image enters a preset shooting trigger area at a first target time, that is, predicting whether the target moving vehicle in the third image enters a preset shooting trigger area after moving for a third time period, may include the following steps:

[0204] S121, determining the current speed and current position of the vehicle in the third image.

[0205] Exemplarily, S121 can refer to the description of S4031 above, which will not be repeated here.

[0206] S122, determining a position P2 of the target moving vehicle in the third image at a first target time according to the current vehicle speed and the current position of the target moving vehicle in the third image.

[0207] Exemplarily, the position P2 of the target moving vehicle in the third image at the first target moment can be determined based on the current speed and current position of the target moving vehicle in the third image, that is, the position P2 of the target moving vehicle in the third image after moving for a third period of time. This can be determined by referring to the description of S4032 above and will not be repeated here.

[0208] S123, determining whether the position P2 of the target moving vehicle in the third image at the first target time belongs to a preset shooting triggering area.

[0209] Exemplarily, when the position P2 of the target moving vehicle in the third image at the first target time belongs to the preset shooting trigger area, it can be determined that the target moving object in the third image enters the preset shooting trigger area at the first target time, that is, the preset trigger condition is met, and S13 is executed. When the position P2 of the target moving vehicle in the third image at the first target time does not belong to the preset shooting trigger area, it can be determined that the target moving object in the third image does not enter the preset shooting trigger area at the first target time, that is, it is determined that the preset trigger condition is not met, and then return to execute S11.

[0210] S13: if the target moving object in the third image enters a preset shooting trigger area at the first target time, second indication information and third indication information are generated.

[0211] For example, when the current angle of the gimbal does not match the position P2 of the target moving vehicle in the third image after the target moving vehicle moves for the third time period, the second image captured by the second image acquisition device at the first target time may not contain the target moving vehicle, or may only contain part of the target moving vehicle. Therefore, before the first target time arrives, the angle of the gimbal can be adjusted to match the position P2 of the target moving vehicle in the third image at the first target time; refer to S21 to S23 below:

[0212] S21, searching preset relationship information according to the position P2 to determine a first angle of the gimbal, where the relationship information includes relationship information between the vehicle position and the gimbal angle.

[0213] For example, assuming that the current position is (x0, y0) and the current speed is V2, the position P2 of the vehicle in the third image after the first period of time is determined to be (x0+V2*(T-t1), y0+V2*(T-t1)).

[0214] Exemplarily, the pre-established relationship information is searched according to the position P2 to determine the angle A1 of the first motor and the angle B1 of the second motor when the vehicle reaches the position P2, that is, the first angle of the gimbal.

[0215] For example, Table 1 may be searched according to position P2 to determine the angle A1 of the first motor and the angle B1 of the second motor when the vehicle reaches position 2.

[0216] For example, x0+V2*(T-t1)=X2, y0+V2*(T-t1)=Y2, then lookup table 1 determines that the angle A1=a2 of the first motor and the angle B1=b2 of the second motor.

[0217] S22, determining a pre-rotation speed of the gimbal according to the current angle of the gimbal, the first angle and the first target time.

[0218] In one possible way, assuming that the current angle of the first motor is A0, the first angle is A1, and the difference between the first target moment and the current moment is the third time length T-t1, the pre-rotation speed VR1 of the first motor can be determined to be (A1-A0) / (T-t1). Assuming that the current angle of the second motor is B0, the first angle is B1, and the difference between the first target moment and the current moment is the third time length T-t1, the pre-rotation speed VR2 of the second motor can be determined to be (B1-B0) / (T-t1).

[0219] In one possible method, assuming that the current angle of the first motor is A0, the first angle is A1, and the difference between the first target moment and the current moment is a third time length T-t1, then the pre-rotation speed VR1 of the first motor can be determined to be (A1-A0) / (T-t1-t4). Assuming that the current angle of the second motor is B0, the first angle is B1, and the difference between the first target moment and the current moment is a third time length T-t1, then the pre-rotation speed VR2 of the second motor can be determined to be (B1-B0) / (T-t1-t4). t4 is less than (T-t1), and can be set specifically according to requirements, and this application does not impose any restrictions on this.

[0220] For example, the pre-rotation speed VR1 of the first motor and the pre-rotation speed VR2 of the second motor are directional. When the value of the pre-rotation speed VR1 of the first motor is positive, its direction can be Figure 2 In the direction in which the first motor rotates toward Ea2, when the value of the pre-rotation speed VR1 of the first motor is negative, its direction can be Figure 2 Of course, it can also be the other way around, that is, when the value of the pre-rotation speed VR1 of the first motor is positive, its direction can be Figure 2In the direction in which the first motor rotates toward Ea1, when the value of the pre-rotation speed VR1 of the first motor is negative, its direction can be Figure 2 When the value of the pre-rotation speed VR2 of the second motor is positive, its direction can be Figure 2 In the direction in which the second motor rotates toward Eb2, when the value of the pre-rotation speed VR2 of the second motor is negative, its direction can be Figure 2 Of course, it can also be the other way around, that is, when the value of the pre-rotation speed VR2 of the second motor is positive, its direction can be Figure 2 In the direction in which the second motor rotates toward Eb1, when the value of the pre-rotation speed VR2 of the second motor is negative, its direction can be Figure 2 The second motor rotates towards Eb2.

[0221] S23, sending fourth instruction information to the pan / tilt head to instruct the pan / tilt head to rotate according to the pre-rotation speed.

[0222] Exemplarily, VR1, VR2 and the waiting time may be used to generate the fourth indication information, and then the fourth indication information is sent to the PTZ, wherein the waiting time is the time between the moment when the PTZ is triggered to rotate and the current moment.

[0223] In one possible manner, when VR1=(A1-A0) / (T-t1), VR2=(B1-B0) / (T-t1), the waiting time may be 0. In this way, after receiving the fourth indication information, the PTZ may immediately control the first motor to rotate according to VR1, and control the second motor to rotate according to VR2.

[0224] In one possible manner, when VR1 = (A1-A0) / (T-t1-t4), VR2 = (B1-B0) / (T-t1-t4), the waiting time may be t4. In this way, after waiting for t4, the PTZ controls the first motor to rotate according to VR1, and controls the second motor to rotate according to VR2.

[0225] Exemplarily, the second indication information may be generated by referring to the following steps S31 to S34.

[0226] S31, predicting the position of the target moving object in the third image at a second target time.

[0227] Exemplarily, the second target moment is after the first target moment, and is separated from the first target moment by a second preset exposure time, that is, the time distance between the second target moment and the current moment is T-t1+t2.

[0228] Exemplarily, the position of the target moving vehicle in the third image at the second target time may be determined so that the second image acquisition device can capture images following the movement of the vehicle.

[0229] Exemplarily, the displacement of the target moving vehicle in the third image at the second target moment may be determined based on the current vehicle speed; and then the position of the target moving vehicle in the third image at the second target moment may be predicted based on the current position and the displacement.

[0230] For example, assuming that the current speed of the vehicle is V2, the predicted displacement of the vehicle after T-t1+t2 is: V2*(T4-t1+t2). If the current position is (x0, y0), the position of the target moving vehicle in the third image at the second target time is (x0, y0+V2*(T4-t1+t2)).

[0231] S32, according to the position of the target moving vehicle in the third image at the second target time, searching for preset relationship information, and determining the target angle of the pan / tilt head, wherein the relationship information includes relationship information between the vehicle position and the pan / tilt head angle.

[0232] For the sake of convenience, the position of the target moving vehicle in the third image at the second target time may be referred to as position P3.

[0233] Exemplarily, pre-established relationship information may be searched according to the position P3 to determine the target angle A2 of the first motor and the target angle B2 of the second motor when the target moving vehicle in the third image is at the position P3.

[0234] Exemplarily, the target angle A2 of the first motor and the target angle B2 of the second motor may be determined by looking up Table 1 according to the position P3 when the target moving vehicle is at the position P3 in the third image.

[0235] For example, x0=X3, y0+V2*(T-t1+t2)=Y3, then lookup table 1 determines that when the target moving vehicle in the third image is at position P3, the target angle A2=a3 of the first motor and the target angle B2=b3 of the second motor.

[0236] S33, determining a target rotation speed of the gimbal according to the current angle of the gimbal, the target angle, and the second preset exposure time.

[0237] Exemplarily, the current angle of the first motor is A1, and the target angle of the first motor is A2, then the target rotation speed of the first motor VR3 = (A2-A1) / t2 can be determined.

[0238] Exemplarily, the current angle of the second motor is B1, and the target angle of the second motor is B2, then the target rotation speed of the second motor VR4=(B2-B1) / t2 can be determined.

[0239] S34, generating second indication information according to the first target time and the target rotation speed.

[0240] Exemplarily, the first target time and the target rotation speed may be used to generate the second indication information; that is, the second indication information may include the first target time and the target rotation speed.

[0241] For example, after receiving the second instruction information, the gimbal can rotate at the target rotation speed at the first target time, that is, control the first motor to rotate at the target rotation speed VR3, and control the second motor to rotate at the target rotation speed VR4. Then, after the first motor and the second motor rotate for the second preset exposure time, the first motor and the second motor are controlled to stop rotating.

[0242] Exemplarily, the third indication information may be generated according to the first target moment and the second preset exposure duration. That is, the third indication information may include the first target moment and the second preset exposure duration.

[0243] Exemplarily, after receiving the third indication information, the second image acquisition device may open the shutter at the first target time, and then close the shutter after exposing the image for the second preset exposure time, thereby ending image acquisition. For ease of explanation, in actual applications of the image processing system, the image acquired by the second image acquisition device at the first target time is referred to as the second image.

[0244] Illustratively, the present application does not limit the order of sending the second indication information and the fourth indication information to the gimbal; nor does it limit the order of sending indication information (including the second indication information and the fourth indication information) to the gimbal, and sending the third indication information to the second image acquisition device.

[0245] Figure 7 FIG. 1 is a schematic diagram showing an exemplary image acquisition process.

[0246] Reference Figure 7 , exemplarily, the first preset exposure time of the first image acquisition device is t1, and the second preset exposure time of the second image acquisition device is t2. After the first image acquisition device acquires the first frame of image, the control device can determine the current position of the vehicle in the first frame of image. According to the current position of the vehicle in the first frame of image, the speed of the vehicle cannot be predicted, and it is also impossible to determine whether the preset trigger condition for triggering the second image acquisition device to start acquisition is met.

[0247] Exemplarily, after the first image acquisition device acquires the second frame of image, the control device may determine the current position of the vehicle in the second frame of image. Then, the current vehicle speed may be determined based on the position of the vehicle license plate in the first frame of image and the position of the vehicle license plate in the second frame of image. Among them, based on the current position and current vehicle speed of the vehicle in the second frame of image, it is determined that the preset trigger condition for triggering the second image acquisition device to start acquisition is not met. Then, the control device continues to determine whether the preset trigger condition for triggering the second image acquisition device to start acquisition is met based on the image acquired by the first image acquisition device.

[0248] Exemplarily, after the first image acquisition device acquires the i-th frame image, the control device determines that the current speed of the vehicle in the i-th frame image is V, and the current position is (x0, y0); and based on the current speed V and current position (x0, y0) of the vehicle predicted by the i-th frame image, it is determined that the vehicle enters the preset shooting trigger area after the third duration of movement, that is, it is determined that the preset trigger condition for triggering the second image acquisition device to start acquisition is met. Then, at F1, the first motor is controlled to rotate according to VR1 and the second motor is controlled to rotate according to VR2 until the first image acquisition device starts to acquire the i+1th frame, that is, at F2, at which time, the angle of the first motor is A1 and the angle of the second motor is B1.

[0249] Continue to refer to Figure 7 For example, at F2 (that is, when the first image acquisition device starts to acquire the i+1th frame), the second image acquisition device is controlled to start acquiring, and the first motor is controlled to rotate according to VR3, and the second motor is controlled to rotate according to VR4. After t2, at F3, the second image acquisition device is controlled to stop exposing, and the first motor is controlled to stop rotating, and the second motor is controlled to stop rotating. At this time, the angle of the first motor is A2, and the angle of the second motor is B2. Then, the second image acquired by the second image acquisition device can be obtained.

[0250] S603, acquiring a second image and a first image acquired by the first image acquisition device at a first target time.

[0251] Exemplarily, for the sake of convenience, the image captured by the first image capture device at the first target time may be referred to as the first image, wherein the first image is the next frame image of the third image.

[0252] Figure 8a and Figure 8b FIG. 1 is a schematic diagram of an image shown as an example.

[0253] in, Figure 8a is a first image captured by a first image acquisition device at a first target time, Figure 8a The user in vehicle H1 cannot be seen.

[0254] Figure 8b a second image acquired by a second image acquisition device, Figure 8b The user in vehicle H2 can be seen.

[0255] S604: Generate a target image according to the first image and the second image.

[0256] Exemplarily, S604 may refer to the following steps S6041 to S6042:

[0257] S6041: Register the second image with the first image according to the target moving object.

[0258] Exemplarily, the second image and the first image may be registered first.

[0259] For example, the target moving vehicle in the second image may be firstly recognized by license plate recognition, and the license plate recognition may be performed on all vehicles in the first image. Then, based on the license plate recognition result of the first image and the license plate recognition result of the second image, the vehicle in the first image that matches the target moving vehicle in the second image is determined, that is, the target moving vehicle in the first image is determined.

[0260] Continue to refer to Figure 8a and Figure 8b , the vehicle in the first image that matches the vehicle H1 in the second image is vehicle H2.

[0261] Exemplarily, the license plate area Z1 (including multiple pixel coordinates) of the target moving vehicle in the second image can be determined, and the license plate area Z2 (including multiple pixel coordinates) of the target moving vehicle in the first image can be determined; then, based on the license plate area Z1 and the license plate area Z2, the coordinate conversion relationship between the target moving vehicle in the first image and the target moving vehicle in the second image can be determined. Exemplarily, the image of the corresponding area of ​​the target moving vehicle can be extracted from the second image to obtain image IMG_C1; then, according to the coordinate conversion relationship, the image IMG_C1 is coordinate-converted to obtain image IMG_C2. After the coordinate conversion of the target moving vehicle in image IMG_C1 is completed, the registration of the second image and the first image is completed.

[0262] S6042, merging the area corresponding to the target moving object in the second image into the area corresponding to the target moving object in the first image

[0263] Exemplarily, the image IMG_C2 may be merged into the first image.

[0264] Exemplarily, the Poisson fusion method is used to fuse the image IMG_C2 and the area corresponding to the target moving vehicle in the first image. Exemplarily, the image IMG_C2 can be divided into K (K is a positive integer) image blocks, and the area corresponding to the target moving vehicle in the first image can be divided into K image blocks. For the i-th (i is a positive integer, ranging from 1 to K, including 1 and K) image block in the K image blocks, the image quality parameters of the i-th image block in the image IMG_C2 and the image quality parameters of the i-th image block in the first image can be compared. Exemplarily, the image quality parameters may include multiple parameters, such as brightness, texture, signal-to-noise ratio, etc., which are not limited in this application. When it is determined that the image quality of the i-th image block in the image IMG_C2 is higher than the image quality of the i-th image block in the second image according to the image quality parameters of the i-th image block in the image IMG_C2 and the image quality parameters of the i-th image block in the second image, the i-th image block in the first image can be replaced with the i-th image block in the image IMG_C2. When it is determined that the image quality of the i-th image block in the first image is higher than the image quality of the i-th image block in the image IMG_C2 according to the image quality parameter of the i-th image block in the first image and the image quality parameter of the i-th image block in the image IMG_C2, it is not necessary to replace the i-th image block in the first image. In this way, an image obtained by fusing the second image with the first image can be obtained.

[0265] Figure 8c FIG. 1 is a schematic diagram of an image shown as an example.

[0266] Reference Figure 8c , exemplary, Figure 8c for Figure 8a and Figure 8b The image obtained after fusion. Figure 8c Users can be seen in vehicle H1.

[0267] Based on the above image processing method, the present application provides an image processing device, which can be deployed in a control device.

[0268] Fig. 9 FIG. 1 is a schematic diagram of the structure of an image processing device shown as an example.

[0269] Reference Fig. 9 , exemplarily, the image processing apparatus 900 comprises:

[0270] The instruction module 901 is used to send first instruction information to the first image acquisition device to instruct the first image acquisition device to acquire images according to a preset frame interval, and the images include the target moving object; send second instruction information to the pan / tilt head to instruct the pan / tilt head to rotate and thereby drive the second image acquisition device to follow the target moving object; and send third instruction information to the second image acquisition device to instruct the second image acquisition device to start acquiring a second image at a first target time, and the second image includes the target moving object;

[0271] An image acquisition module 902 is used to acquire a second image and a first image acquired by a first image acquisition device at a first target time;

[0272] An image generation module 903 is used to generate a target image according to the first image and the second image, where the target image includes a target moving object;

[0273] Among them, the first preset exposure time of the first image acquisition device is smaller than the second preset exposure time of the second image acquisition device.

[0274] Exemplarily, the image generation module 903 is specifically used to register the first image and the second image according to the target moving object; and fuse the area corresponding to the target moving object in the second image to the area corresponding to the target moving object in the first image.

[0275] Continue Parameters Fig. 9 , illustratively, the image processing device 900 further includes:

[0276] The indication generation module 904 is used to obtain a third image, which is acquired by the first image acquisition device and is the previous frame image of the first image; predict whether the target moving object in the third image enters a preset shooting trigger area at the first target moment; if it enters the preset shooting trigger area, generate second indication information and generate third indication information.

[0277] Exemplarily, the indication generating module 904 is specifically used to predict the first position of the target moving object in the third image at the second target moment, wherein the second target moment is after the first target moment and is a second preset exposure time away from the first target moment; searching for preset relationship information according to the first position to determine the target angle of the gimbal, wherein the relationship information includes relationship information between the position of the moving object and the angle of the gimbal; determining the target rotation speed of the gimbal according to the current angle of the gimbal, the target angle, and the second preset exposure time; generating the second indication information according to the first target moment and the target rotation speed

[0278] Exemplarily, the indication generating module 904 is specifically configured to generate third indication information according to the first target time and the second preset exposure duration.

[0279] In one example, Fig.10 A schematic block diagram of a device 1000 according to an embodiment of the present application is shown. The device 1000 may include: a processor 1001 and a transceiver / transceiver pin 1002 , and optionally, a memory 1003 .

[0280] The various components of the device 1000 are coupled together via a bus 1004, wherein the bus 1004 includes a power bus, a control bus, and a status signal bus in addition to a data bus. However, for the sake of clarity, various buses are referred to as bus 1004 in the figure.

[0281] Optionally, the memory 1003 may be used for instructions in the aforementioned method embodiment. The processor 1001 may be used to execute instructions in the memory 1003, and control the receiving pin to receive a signal, and control the sending pin to send a signal.

[0282] The apparatus 1000 may be the electronic device or a chip of the electronic device in the above method embodiment.

[0283] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here.

[0284] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the image processing method in the above-mentioned embodiment.

[0285] This embodiment further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the image processing method in the above-mentioned embodiment.

[0286] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the image processing method in the above-mentioned method embodiments.

[0287] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0288] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0289] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0290] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0291] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0292] Any content of each embodiment of the present application, as well as any content of the same embodiment, can be freely combined. Any combination of the above content is within the scope of the present application.

[0293] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0294] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0295] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing a software instruction. The software instruction can be composed of corresponding software modules, and the software module can be stored in a random access memory (Random Access Memory, RAM), a flash memory, a read-only memory (Read Only Memory, ROM), an erasable programmable read-only memory (Erasable Programmable ROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disk (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0296] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0297] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Sending first instruction information to a first image acquisition device to instruct the first image acquisition device to acquire an image according to a preset frame interval, wherein the image includes a target moving object; Sending second instruction information to the pan-tilt head to instruct the pan-tilt head to rotate and thereby drive the second image acquisition device to follow the target moving object; and sending third instruction information to the second image acquisition device to instruct the second image acquisition device to start acquiring a second image at a first target time, wherein the second image includes the target moving object; Acquire the second image and the first image acquired by the first image acquisition device at the first target time; Generate a target image according to the first image and the second image, wherein the target image includes the target moving object; Among them, the first preset exposure time of the first image acquisition device is smaller than the second preset exposure time of the second image acquisition device.

2. The method according to claim 1, characterized in that The method further comprises: Acquire a third image, where the third image is acquired by the first image acquisition device and is a frame image previous to the first image; Predicting whether the target moving object in the third image enters a preset shooting trigger area at the first target moment; If entering the shooting trigger area, the second indication information is generated and the third indication information is generated.

3. The method according to claim 2, characterized in that The generating the second indication information includes: Predicting a position of the target moving object in the third image at a second target moment, wherein the second target moment is after the first target moment and is separated from the first target moment by the second preset exposure time; Searching preset relationship information according to the position to determine the target angle of the pan / tilt platform, wherein the relationship information includes relationship information between the position of the moving object and the pan / tilt platform angle; Determining a target rotation speed of the gimbal according to the current angle of the gimbal, the target angle, and the second preset exposure time; The second indication information is generated according to the first target time and the target rotation speed.

4. The method according to claim 2 or 3, characterized in that: The generating the third indication information includes: The third indication information is generated according to the first target moment and the second preset exposure duration.

5. The method according to any one of claims 1 to 3, characterized in that: The step of generating a target image according to the second image and the first image acquired by the first image acquisition device at the first target time comprises: According to the target moving object, registering the first image and the second image; The area corresponding to the target moving object in the second image is fused to the area corresponding to the target moving object in the first image.

6. The method according to any one of claims 1 to 3, characterized in that: The target moving object in the second image is in a preset shooting area.

7. The method according to any one of claims 1 to 3, characterized in that: The resolution of the target image is the same as the resolution of the first image.

8. The method according to any one of claims 1 to 3, characterized in that: The pan-tilt platform is a two-dimensional pan-tilt platform; The rotation direction of the pan / tilt head includes: a horizontal direction and a pitch direction.

9. The method according to any one of claims 1 to 3, characterized in that: The focal length of the first image acquisition device is smaller than the focal length of the second image acquisition device.

10. An image processing system, characterized in that: The image acquisition system comprises: a first image acquisition device, a pan / tilt head and a second image acquisition device; wherein, The first image acquisition device is used to acquire images according to a preset frame interval and using a first preset exposure time; The pan / tilt platform is used to support the second image acquisition device and drive the second image acquisition device to rotate; The second image acquisition device is used to acquire images using a second preset exposure time; Among them, the first preset exposure time of the first image acquisition device is smaller than the second preset exposure time of the second image acquisition device.

11. The image acquisition system according to claim 10, characterized in that: The image acquisition system also includes a control device; The control device communicates with the first image acquisition device and the second image acquisition device through wired communication or wireless communication to control the first image acquisition device and the second image acquisition device to acquire images; The control device communicates with the pan / tilt platform via wired communication or wireless communication to control the rotation of the pan / tilt platform.

12. An image processing device, characterized in that: The device comprises: an instruction module, configured to send first instruction information to a first image acquisition device to instruct the first image acquisition device to acquire an image according to a preset frame interval, wherein the image includes a target moving object; send second instruction information to a pan / tilt head to instruct the pan / tilt head to rotate and thereby drive the second image acquisition device to follow the movement of the target moving object; and send third instruction information to the second image acquisition device to instruct the second image acquisition device to start acquiring a second image at a first target time, wherein the second image includes the target moving object; An image acquisition module, used to acquire the second image and the first image acquired by the first image acquisition device at the first target time; An image generation module, configured to generate a target image according to the first image and the second image, wherein the target image includes the target moving object; Among them, the first preset exposure time of the first image acquisition device is smaller than the second preset exposure time of the second image acquisition device.

13. An electronic device, characterized in that: include: a memory and a processor, the memory being coupled to the processor; The memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the image processing method according to any one of claims 1 to 9.

14. A chip system, characterized in that: It comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal, and the signal includes a computer instruction; when the one or more processors execute the computer instruction, the electronic device executes the image processing method as described in any one of claims 1 to 9.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program runs on a computer or a processor, the computer or the processor executes the image processing method according to any one of claims 1 to 9.

16. A computer program product comprising instructions, characterized in that When the image processing method is executed on a computer, the computer is enabled to execute the image processing method according to any one of claims 1 to 9.

Images