Network camera holder control method and device for object movement tracking, electronic equipment and computer readable medium

Abstract

The embodiment of the invention discloses a network camera holder control method and device for object movement tracking, electronic equipment and a computer readable medium. A specific embodiment of the method comprises the following steps: performing image segmentation on each scene image in a scene image sequence in a scene video; performing motion detection on each segmented scene image in the segmented scene image set sequence; performing association processing on each segmented scene image in each segmented scene image set in the segmented scene image set sequence to generate an image communication region, and obtaining an image communication region set; for each image communication region in the image communication region set, determining the image communication region as the region where the moving object is located; determining a region center point coordinate of the image communication region, and determining an image center point coordinate of the corresponding scene image; and controlling the pan-tilt of the network camera to rotate so as to perform object movement tracking. According to the embodiment, the occupation of computing resources in the network camera can be reduced.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of object tracking, and in particular to a network camera pan / tilt control method, device, electronic device, and computer-readable medium for object movement tracking. Background Art

[0002] Object motion tracking is a technology used to identify and follow the movement of objects in the camera. Currently, the method commonly used for object motion tracking is to use a neural network model to identify moving objects in the captured video and predict the direction of the object's movement for tracking.

[0003] However, in practice, it is found that when the above method is used to generate obstacle information, the following technical problems often occur:

[0004] In IPC (Internet Protocol Camera) devices (when deployed with small memory), the recognition of moving objects and the prediction of the direction of moving objects through neural network models require more computing resources, which in turn affects the operation of other functions of the device.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose a network camera pan-tilt control method, device, electronic device and computer-readable medium for object movement tracking to solve the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a method for controlling a webcam pan / tilt for object motion tracking, the method comprising: acquiring a scene video captured by a webcam, wherein the scene video includes a scene image sequence of continuous frames; performing image segmentation on each scene image in the scene image sequence in the scene video to generate a sequence of segmented scene image sets; performing motion detection on each segmented scene image in the segmented scene image set sequence by an image processing module deployed in a device chip in the webcam to generate a sequence of scene image motion detection value sets; based on the scene image motion detection value set sequence, performing motion detection on each segmented scene image in the segmented scene image set sequence. The segmented scene images are associated with each other to generate an image connection area, and an image connection area set is obtained, wherein the image connection area is the largest associated area in the segmented scene image; for each image connection area in the above image connection area set, the following control steps are performed: in response to determining that the area ratio of the above image connection area to the area of ​​the corresponding scene image is greater than or equal to a preset area ratio threshold, the image connection area is determined to be the area where the moving object is located; the coordinates of the area center point of the above image connection area are determined, and the coordinates of the image center point of the corresponding scene image are determined; based on the coordinates of the area center point and the image center point, the pan / tilt of the above network camera is controlled to rotate to track the movement of the object.

[0009] In a second aspect, some embodiments of the present disclosure provide a network camera pan-tilt control device for object movement tracking, the device comprising: an acquisition unit, configured to acquire a scene video captured by a network camera, wherein the scene video includes a scene image sequence of continuous frames; an image segmentation unit, configured to perform image segmentation on each scene image in the scene image sequence in the scene video to generate a segmented scene image set sequence; a motion detection unit, configured to perform motion detection on each segmented scene image in the segmented scene image set sequence through an image processing module deployed in a device chip in the network camera to generate a scene image motion detection value set sequence; an image association unit, configured to associate the segmented scene images with the scene image set sequence based on the scene image motion detection value set sequence. Each segmented scene image in each segmented scene image set in the image set sequence is associated to generate an image connection area, and an image connection area set is obtained, wherein the image connection area is the largest associated area in the segmented scene image; the pan / tilt control unit is configured to perform the following control steps for each image connection area in the image connection area set: in response to determining that the area ratio of the image connection area to the area of ​​the corresponding scene image is greater than or equal to a preset area ratio threshold, determine the image connection area as the area where the moving object is located; determine the coordinates of the area center point of the image connection area, and determine the coordinates of the image center point of the corresponding scene image; based on the coordinates of the area center point and the image center point, control the pan / tilt of the network camera to rotate to track the movement of the object. .

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0012] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the network camera pan-tilt control method for object movement tracking of some embodiments of the present disclosure, the occupation of computing resources can be reduced. Specifically, the reason why more computing resources need to be occupied is that in the IPC (Internet Protocol Camera, network camera) device (when the deployment memory is small), the recognition of the moving object by the neural network model is combined with the prediction of the direction of the moving object. Based on this, the network camera pan-tilt control method for object movement tracking of some embodiments of the present disclosure first obtains the scene video shot by the network camera. Among them, the above-mentioned scene video includes a scene image sequence of continuous frames. Then, each scene image in the scene image sequence in the above-mentioned scene video is image segmented to generate a segmented scene image set sequence. Here, by image segmentation, it is convenient to detect moving objects in different regions. Then, through the image processing module deployed in the device chip in the above-mentioned network camera, each segmented scene image in the segmented scene image set sequence is subjected to motion detection to generate a scene image motion detection value set sequence. Here, by using the image processing module built into the network camera device chip, the scene image motion detection value corresponding to each segmented scene image can be quickly determined. Next, based on the above-mentioned scene image motion detection value set sequence, each segmented scene image in each segmented scene image set in the above-mentioned segmented scene image set sequence is subjected to association processing to generate an image connection area, and an image connection area set is obtained. Among them, the image connection area is the largest associated area in the segmented scene image. Here, through the association processing, multiple areas corresponding to the moving object can be associated. Also because the scene image motion detection value is generated, it can be used to quickly locate the area where the moving object is located. Therefore, there is no need to perform overall detection of the moving object through the neural network model. Finally, for each image connection area in the above-mentioned image connection area set, the following control steps are performed: First, in response to determining that the area ratio of the above-mentioned image connection area to the area ratio of the corresponding scene image is greater than or equal to a preset area ratio threshold, the above-mentioned image connection area is determined to be the area where the moving object is located. Among them, by introducing a preset area ratio threshold, it can be used to exclude the detection of moving objects occupying a smaller image area. Not only can it avoid false detection caused by fewer features, but it can also avoid affecting the detection of moving objects in other areas. Make it possible to detect moving objects that are closer as much as possible. Second, determine the coordinates of the center point of the region of the image connection area, and determine the coordinates of the center point of the image of the corresponding scene image. Third, based on the coordinates of the center point of the region and the coordinates of the center point of the image, control the pan / tilt of the network camera to rotate to track the movement of the object. Thus, the network camera pan / tilt control method for object movement tracking in some embodiments of the present disclosure can rely on the image processing module of the IPC device chip to output the motion detection value for the identification of the moving object.Thus, it is avoided to deploy an algorithm module designated for object movement detection in the IPC device. Therefore, the identification and tracking of moving objects can be achieved only by the above method. This reduces the occupation of computing resources. Furthermore, it can also ensure the allocation of computing resources for other functions in the device, avoiding the impact of computing resource allocation problems on the operation of other functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0014] Figure 1 is a flow chart of some embodiments of a webcam pan / tilt control method for object motion tracking according to the present disclosure;

[0015] Figure 2 is a schematic structural diagram of some embodiments of a network camera pan / tilt control device for object movement tracking according to the present disclosure;

[0016] Figure 3 is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure;

[0017] Figure 4 is a schematic diagram of the structure of some embodiments according to the present disclosure;

[0018] Figure 5 is another structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0020] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0021] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0022] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0024] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0025] Figure 1 The process 100 of some embodiments of the network camera pan-tilt control method for object movement tracking according to the present disclosure is shown. The network camera pan-tilt control method for object movement tracking includes the following steps:

[0026] Step 101, obtaining a scene video captured by a network camera.

[0027] In some embodiments, the execution subject of the network camera pan-tilt control method for object movement tracking can obtain the scene video shot by the network camera in a wired or wireless manner. The scene video may include a scene image sequence of continuous frames.

[0028] It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0029] Step 102 : performing image segmentation on each scene image in the scene image sequence in the scene video to generate a segmented scene image set sequence.

[0030] In some embodiments, the execution entity may perform image segmentation on each scene image in the scene image sequence in the scene video to generate a segmented scene image set sequence.

[0031] In some optional implementations of some embodiments, the execution subject performs image segmentation on each scene image in the scene image sequence in the scene video to generate a segmented scene image set sequence, which may include the following steps:

[0032] For each scene image in the scene image sequence, the scene image is segmented according to the preset segmentation data to generate a segmented scene image set. The sizes of the segmented scene images in the segmented scene image set are the same. For example, the preset segmentation data may be a 32×24 pixel area. Thus, each scene image may be evenly segmented into 32×24 segmented scene images of the same size by the image segmentation function.

[0033] In practice, when tracking the movement of an object, a method that can usually be used is to deploy an algorithm for background modeling in an IPC device, and identify moving objects in an image by background modeling. For example, median method background modeling, mean method background modeling, single Gaussian distribution model, mixed Gaussian distribution model, Kalman filter model, and advanced background model, etc. However, these methods are all based on the brightness value of the pixel for mathematical calculation processing. However, since the brightness of the scene often changes, the modeled background needs to be adjusted frequently. This also requires more computing resources. For example, background modeling is performed through a mixed Gaussian distribution model. Although it can be highly adaptable to complex scenes, and the background model can be adjusted through automatically calculated model parameters. The purpose of improving detection speed and detection accuracy is achieved. However, the number of Gaussian distributions is increased, resulting in an increase in the amount of calculation. Therefore, the above-mentioned technical problems also exist.

[0034] Optionally, the execution subject performs image segmentation on each scene image in the scene image sequence in the scene video to generate a sequence of segmented scene image sets, which may include the following steps: the first step is to obtain the shooting angle of each scene image in the scene video shot by the network camera to obtain a shooting angle sequence. The shooting angle may be the direction of the straight line where the optical axis of the network camera is located, which may be represented by a vector.

[0035] In the second step, for each scene image and the corresponding shooting angle in the above scene image sequence, the following processing steps are performed:

[0036] Step 1, obtain the preset background slice area coordinate group corresponding to the above shooting angle, and obtain a background slice coordinate group set. Each preset background slice coordinate group can correspond to a background area. In practice, in the direction of each shooting angle, the network camera can capture a certain area, then the background area that intersects with the area is the background area corresponding to the above shooting angle. Therefore, the preset background slice area coordinate group corresponding to the background area can be determined as the background slice coordinate group, and a background slice coordinate group set is obtained.

[0037] Step 2: segment the scene image according to the area of ​​each background slice coordinate in each background slice coordinate group in the background slice coordinate group set to generate a segmented scene image set. The segmentation can be performed along each background slice coordinate in each background slice coordinate group by an image segmentation function to generate a segmented scene image set.

[0038] Optionally, the above-mentioned preset background slice coordinate group is generated by the following steps:

[0039] The first step is to control the network camera to perform empty shot shooting according to a preset shooting trajectory to obtain a scene sample background image sequence. The shooting trajectory can be the pan / tilt movement trajectory of the network camera. Each scene sample background image in the scene sample background image sequence shot according to the shooting trajectory fully covers the area that can be shot by the network camera. Empty shot shooting can be shooting without people or moving objects in the video. The scene sample background image sequence can be images in the same video or images in multiple videos.

[0040] The second step is to perform regional segmentation on each scene sample image in the above scene sample background image sequence to generate a sample image segmentation region set. Among them, the region segmentation of each scene sample image in the above scene sample background image sequence can be performed by a pre-trained network model to generate a sample image segmentation region set. Here, the generation of the preset background slice coordinate group can be generated by the network camera server according to the scene sample background image sequence. For example, the network model can be a YOLO-v5 (You Only Look Once-Version5) network model, a VGG (Visual Geometry Group Network, convolutional neural network) model, etc.

[0041] The third step is to perform region stitching processing on the above sample image segmentation region set to generate a stitched region atlas set. The region stitching processing can stitch the sample image segmentation regions corresponding to the same target region in different scene sample background images through a preset stitching algorithm. For example, the stitching algorithm can include but is not limited to at least one of the following: SIFT (Scale-invariant Feature Transform) algorithm, Surf (Speeded Up Robust Features) algorithm.

[0042] The fourth step is to filter the above-mentioned spliced ​​area atlas set according to the preset filtering conditions to obtain the target sample image segmentation area set. The above-mentioned filtering conditions can be used to determine that each target sample image segmentation area in the above-mentioned target sample image segmentation area set fully covers the area that can be photographed by the above-mentioned network camera. The filtering condition can be that there are no excessive repeated areas between the selected target sample image segmentation areas, and they can fully cover the area that can be photographed by the above-mentioned network camera. Therefore, it can be used to model the background reference image for image segmentation.

[0043] The fifth step is to determine the sample shooting angle interval corresponding to each sample image segmentation area in the above sample image segmentation area set to obtain a sample shooting angle interval set. The sample shooting angle area can represent the shooting angle of a network camera corresponding to a sample image segmentation area. The sample shooting angle interval can be used to associate the correspondence between the sample image segmentation area and the shooting angle. Here, the shooting angle can be represented by a (three-dimensional) vector. Then the shooting angle interval can be the area of ​​data in the vector.

[0044] In the sixth step, the region boundary coordinates of each target sample image segmentation region in the target sample image segmentation region set are determined as a preset background slice coordinate group.

[0045] The above optional steps and their related contents, as an inventive point of an embodiment of the present disclosure, can further solve the technical problem of "occupying more computing resources" mentioned in the background technology. In order to achieve this effect, first of all, considering the influence of brightness on background modeling, the above implementation method can associate different background areas through shooting angles. In this way, there is no need to consider brightness, and the corresponding preset background slice area coordinate group stored in advance can be selected according to the shooting angle of the network camera. This is used as the background area of ​​the shooting area. Here, since the background area is a static area and is pre-divided according to different static objects. Then the scene image can be segmented according to each preset background slice area coordinate. As a result, each segmented area is a fixed area where a static object is located. Therefore, it is convenient to determine whether there is a pixel change in the area and the specific situation of the pixel change. Then, it is determined whether there is a moving object. At the same time, in order to further reduce the occupation of computing resources, the server pre-segmented the scene sample background image sequence corresponding to the collected empty shot video to achieve the segmentation of the background area. At the same time, by introducing the sample shooting angle interval, it is used to establish multiple target sample image segmentation areas corresponding to different shooting angles of the network camera. Thereby, it is convenient for the IPC device to directly call during the object motion detection process. Therefore, compared with the above-mentioned background modeling method, background modeling can be achieved without adaptively updating the image background according to brightness, thereby reducing the consumption of computing resources.

[0046] Step 103 , performing motion detection on each segmented scene image in the segmented scene image set sequence by using an image processing module deployed in a device chip in the network camera, so as to generate a scene image motion detection value set sequence.

[0047] In some embodiments, the execution subject may perform motion detection on each segmented scene image in the segmented scene image set sequence through an image processing module deployed in a device chip in the network camera to generate a scene image motion detection value set sequence. The image processing module may include a pre-set detection algorithm, such as a frame difference method. The scene image motion detection value of the segmented scene image is generated by this detection. Here, the scene image motion detection value may be a probability value or a specific data value.

[0048] In some optional implementations of some embodiments, the execution subject performs motion detection on each segmented scene image in the segmented scene image set sequence to generate a scene image motion detection value set sequence, which may include the following steps:

[0049] In the first step, the segmented scene images at corresponding positions in two adjacent segmented scene image sets in the above segmented scene image set sequence are compared pixel by pixel to generate a total pixel difference value, thereby obtaining a total pixel difference value set. The sum of the pixel values ​​in each of the two adjacent segmented scene images can be determined first. Then, the sum of the two pixel values ​​can be subtracted to obtain the total pixel difference value.

[0050] In the second step, a total pixel difference set between each segmented scene image set in the segmented scene image set sequence and an adjacent previous segmented scene image set is determined as a scene image motion detection value set to obtain a scene image motion detection value set sequence. For the first segmented scene image set in the segmented scene image set sequence, the corresponding scene image motion detection value set may not be output.

[0051] Step 104 , based on the scene image motion detection value set sequence, performing association processing on each segmented scene image in each segmented scene image set in the segmented scene image set sequence to generate an image connected area, thereby obtaining an image connected area set.

[0052] In some embodiments, the execution subject may perform association processing on each segmented scene image in each segmented scene image set in the segmented scene image set sequence based on the scene image motion detection value set sequence to generate an image connection area, thereby obtaining an image connection area set. The image connection area may be the largest association area in the segmented scene image.

[0053] In some optional implementations of some embodiments, the execution subject may perform association processing on each segmented scene image in each segmented scene image set in the segmented scene image set sequence based on the scene image motion detection value set sequence to generate an image connection area, which may include the following steps:

[0054] In the first step, segmented scene images whose corresponding scene image motion detection values ​​are greater than a preset detection threshold are selected from the segmented scene image set to generate a target scene image group.

[0055] In the second step, each target scene image in the target scene image group is regionally connected to generate an image connection area. Among them, each target scene image in the target scene image group can be regionally connected to generate an image connection area through a preset regional connection algorithm. Here, in the regional connection process, the regional connection algorithm can use the largest area after connection as the image connection area. For example, the regional connection algorithm can be a Two-Pass (two-pass scanning method) two-pass scanning method.

[0056] Step 105, for each image connected area in the image connected area set, execute the following control steps:

[0057] Step 1051, in response to determining that the area ratio of the image connection area to the area ratio of the corresponding scene image is greater than or equal to a preset area ratio threshold, determining that the image connection area is an area where the moving object is located.

[0058] In some embodiments, the execution subject may determine that the image connection area is the area where the moving object is located in response to determining that the area ratio of the image connection area to the corresponding scene image is greater than or equal to a preset area ratio threshold. Among them, each image connection area can determine the one-to-one correspondence between the scene image and the corresponding segmented scene image set. In practice, the preset area ratio threshold may be 1 / 18. If it is greater than this value, it can be determined that there is a moving object that needs to be tracked in the image connection area.

[0059] Step 1052, determining the coordinates of the center point of the image connection area, and determining the coordinates of the center point of the corresponding scene image.

[0060] In some embodiments, the execution subject may determine the coordinates of the regional center point of the image connection area and the image center point coordinates of the corresponding scene image. The coordinates of the regional center point of the image connection area and the image center point coordinates of the corresponding scene image may be determined by a preset function. For example, the cvMoments function of OpenCV may be used to calculate the moment information, and then the cvGetSpatialMoment function may be used to obtain the centroid coordinates as the image center point coordinates. Secondly, the coordinates of the image center point may be used as the image center point coordinates.

[0061] Step 1053, based on the coordinates of the center point of the region and the coordinates of the center point of the above-mentioned image, control the pan / tilt of the network camera to rotate to track the movement of the object.

[0062] In some embodiments, the execution entity may control the pan / tilt of the network camera to rotate based on the coordinates of the center point of the region and the coordinates of the center point of the image to track the movement of the object.

[0063] In some optional implementations of some embodiments, the execution subject controls the pan / tilt of the network camera to rotate based on the coordinates of the center point of the region and the coordinates of the center point of the image to track the movement of the object, which may include the following steps:

[0064] The first step is to determine the area ratio of the above image connection area in the corresponding scene image. Among them, first, the minimum rectangular frame of the above image connection area can be determined. Then, the ratio between the number of pixels in the minimum rectangular frame and the number of pixels in the scene image can be determined as the area ratio.

[0065] The second step is to determine the pixel distance value between the coordinates of the center point of the region and the coordinates of the center point of the image. The pixel distance value between the coordinates of the center point of the region and the coordinates of the center point of the image can be determined by a preset distance algorithm. For example, the pixel distance value can be determined by Euclidean distance.

[0066] The third step is to control the pan / tilt of the network camera to rotate to the left in response to determining that the horizontal coordinate value of the midpoint coordinates of the above-mentioned area is less than the horizontal coordinate value of the coordinates of the center point of the above-mentioned image, the area proportion is greater than the preset area proportion threshold, and the pixel distance value is greater than the preset distance threshold. Wherein, when the horizontal coordinate value of the midpoint coordinates of the above-mentioned area is equal to the horizontal coordinate value of the coordinates of the center point of the above-mentioned image, no left or right rotation is performed. In practice, the horizontal coordinate value of the midpoint coordinates of the area is less than the horizontal coordinate value of the coordinates of the center point of the above-mentioned image, which can indicate that the image connection area (i.e., the image area where the moving object is located) is biased to the left side of the image. Therefore, when the area proportion is greater than the preset area proportion threshold (for example, the preset area proportion threshold can be a value between 1 / 18-1 / 14), and the pixel distance value is greater than the preset distance threshold (for example, 50 pixels), the pan / tilt of the network camera can be controlled to rotate to the left to achieve the purpose of tracking the moving object.

[0067] In the fourth step, in response to determining that the horizontal coordinate value of the midpoint coordinates of the above region is greater than the horizontal coordinate value of the center point coordinates of the above image, the area ratio is greater than the preset area ratio threshold, and the pixel distance value is greater than the preset distance threshold, the pan / tilt of the above network camera is controlled to rotate right. In practice, the horizontal coordinate value of the midpoint coordinates of the above region is greater than the horizontal coordinate value of the center point coordinates of the above image, which can indicate that the image connection area is biased to the right side of the image. Therefore, when the area ratio is greater than the preset area ratio threshold, and the pixel distance value is greater than the preset distance threshold, the pan / tilt of the network camera can be controlled to rotate left.

[0068] The fifth step is to control the pan / tilt of the network camera to rotate downward in response to determining that the ordinate value of the midpoint coordinates of the above region is less than the ordinate value of the coordinates of the center point of the above image, and the pixel distance value is greater than a preset distance threshold. Wherein, when the ordinate value of the midpoint coordinates of the above region is less than the ordinate value of the coordinates of the center point of the above image, no up and down rotation is performed. In practice, the ordinate value of the midpoint coordinates of the region is less than the ordinate value of the coordinates of the center point of the above image, which can indicate that the image connection area is biased toward the lower side of the image (with the pixel in the lower left corner of the image as the first pixel). Therefore, the pan / tilt of the network camera is controlled to rotate downward.

[0069] In step 6, in response to determining that the vertical coordinate value of the coordinate of the midpoint of the region is greater than the vertical coordinate value of the coordinate of the center point of the image, and the pixel distance value is greater than a preset distance threshold, the pan / tilt of the network camera is controlled to rotate upward. The distances between the horizontal and vertical coordinate values ​​of the coordinate of the center point of the region and the coordinate of the center point of the image are different, and the pan / tilt is controlled to rotate at different angles in different directions.

[0070] In practice, if the ordinate value of the midpoint coordinates of the above-mentioned area is greater than the ordinate value of the coordinates of the center point of the above-mentioned image, it can be characterized that the image connection area is biased towards the upper side of the image. Therefore, the pan / tilt of the above-mentioned network camera is controlled to rotate upward. In addition, if the area ratio is greater than the preset area ratio threshold, it can be characterized that the picture area is larger (the corresponding distance is closer), and motion detection can be performed. If the pixel distance value is greater than the preset distance threshold, it can be characterized that the area is offset more, so it is necessary to rotate the pan / tilt to drive the network camera for motion detection. Specifically, the pan / tilt moves 160 to 255 steps in the horizontal movement direction (one step defaults to 0.5% angle, which can be customized as needed). In the vertical movement direction of the pan / tilt, if the upper side is within 45 pixels, move up 25 steps; if the upper side is below 160 pixels, move down 100 steps.

[0071] Optionally, the above execution entity may further perform the following steps:

[0072] The first step is to determine the object moving direction vector based on the positional relationship between the coordinates of the center point of the region and the coordinates of the center point of the image, wherein the vector between the coordinates of the center point of the image and the coordinates of the center point of the region can be determined as the object moving direction vector.

[0073] The second step is to render the object moving direction vector to the corresponding position in each scene image in the scene image sequence, and mark the object detection frame in each scene image to obtain a rendered scene image sequence. The rendering may be to add the object moving direction vector to the position of the corresponding region center point coordinates and image center point coordinates in each scene image. In addition, the object detection frame may also be marked for user viewing.

[0074] The third step is to send the rendered scene image sequence to a display terminal for display.

[0075] Optionally, the execution subject may also determine that no moving object is detected in the scene video in response to determining that the ratio of the area of ​​the image connection region to the area of ​​the corresponding scene image is less than a preset area ratio threshold.

[0076] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the network camera pan-tilt control method for object movement tracking of some embodiments of the present disclosure, the occupation of computing resources can be reduced. Specifically, the reason why more computing resources need to be occupied is that in the IPC (Internet Protocol Camera, network camera) device (when the deployment memory is small), the recognition of the moving object by the neural network model is combined with the prediction of the direction of the moving object. Based on this, the network camera pan-tilt control method for object movement tracking of some embodiments of the present disclosure first obtains the scene video shot by the network camera. Among them, the above-mentioned scene video includes a scene image sequence of continuous frames. Then, each scene image in the scene image sequence in the above-mentioned scene video is image segmented to generate a segmented scene image set sequence. Here, by image segmentation, it is convenient to detect moving objects in different regions. Then, through the image processing module deployed in the device chip in the above-mentioned network camera, each segmented scene image in the segmented scene image set sequence is subjected to motion detection to generate a scene image motion detection value set sequence. Here, by using the image processing module built into the network camera device chip, the scene image motion detection value corresponding to each segmented scene image can be quickly determined. Next, based on the above-mentioned scene image motion detection value set sequence, each segmented scene image in each segmented scene image set in the above-mentioned segmented scene image set sequence is subjected to association processing to generate an image connection area, and an image connection area set is obtained. Among them, the image connection area is the largest associated area in the segmented scene image. Here, through the association processing, multiple areas corresponding to the moving object can be associated. Also because the scene image motion detection value is generated, it can be used to quickly locate the area where the moving object is located. Therefore, there is no need to perform overall detection of the moving object through the neural network model. Finally, for each image connection area in the above-mentioned image connection area set, the following control steps are performed: First, in response to determining that the area ratio of the above-mentioned image connection area to the area ratio of the corresponding scene image is greater than or equal to a preset area ratio threshold, the above-mentioned image connection area is determined to be the area where the moving object is located. Among them, by introducing a preset area ratio threshold, it can be used to exclude the detection of moving objects occupying a smaller image area. Not only can it avoid false detection caused by fewer features, but it can also avoid affecting the detection of moving objects in other areas. Make it possible to detect moving objects that are closer as much as possible. Second, determine the coordinates of the center point of the region of the image connection area, and determine the coordinates of the center point of the image of the corresponding scene image. Third, based on the coordinates of the center point of the region and the coordinates of the center point of the image, control the pan / tilt of the network camera to rotate to track the movement of the object. Thus, the network camera pan / tilt control method for object movement tracking in some embodiments of the present disclosure can rely on the image processing module of the IPC device chip to output the motion detection value for the identification of the moving object.Thus, it is avoided to deploy an algorithm module designated for object movement detection in the IPC device. Therefore, the identification and tracking of moving objects can be achieved only by the above method. This reduces the occupation of computing resources. Furthermore, it can also ensure the allocation of computing resources for other functions in the device, avoiding the impact of computing resource allocation problems on the operation of other functions.

[0077] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a network camera pan-tilt control device for object movement tracking, and these device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0078] like Figure 2 As shown, in some embodiments, a network camera pan-tilt control device 200 for object movement tracking includes: an acquisition unit 201, an image segmentation unit 202, a motion detection unit 203, an image association unit 204, and a pan-tilt control unit 205. The acquisition unit is configured to acquire a scene video shot by a network camera, wherein the scene video includes a scene image sequence of continuous frames; the image segmentation unit is configured to perform image segmentation on each scene image in the scene image sequence in the scene video to generate a segmented scene image set sequence; the motion detection unit is configured to perform motion detection on each segmented scene image in the segmented scene image set sequence through an image processing module deployed in a device chip in the network camera to generate a scene image motion detection value set sequence; the image association unit is configured to perform motion detection on each segmented scene image in each segmented scene image set in the segmented scene image set sequence based on the scene image motion detection value set sequence. The scene image is associated to generate an image connection area, and an image connection area set is obtained, wherein the image connection area is the largest associated area in the scene image after segmentation; the pan-tilt control unit is configured to perform the following control steps for each image connection area in the above image connection area set: in response to determining that the area ratio of the above image connection area to the area of ​​the corresponding scene image is greater than or equal to a preset area ratio threshold, determine the above image connection area as the area where the moving object is located; determine the coordinates of the area center point of the above image connection area, and determine the coordinates of the image center point of the corresponding scene image; based on the coordinates of the area center point and the coordinates of the image center point, control the pan-tilt of the above network camera to rotate to track the movement of the object. .

[0079] It is understood that the units described in the device 200 are similar to those described in the reference Figure 1Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 200 and the units included therein, and will not be described in detail here.

[0080] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device (eg, a computing device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0081] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory 302 or a program loaded from a storage device 308 into a random access memory 303. Various programs and data required for the operation of the electronic device 300 are also stored in the random access memory 303. The processing device 301, the read-only memory 302, and the random access memory 303 are connected to each other via a bus 304. An input / output interface 305 is also connected to the bus 304.

[0082] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0083] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the read-only memory 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0084] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0085] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0086] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains a scene video captured by a network camera, wherein the above-mentioned scene video includes a scene image sequence of continuous frames; performs image segmentation on each scene image in the scene image sequence in the above-mentioned scene video to generate a segmented scene image set sequence; performs motion detection on each segmented scene image in the segmented scene image set sequence through an image processing module deployed in the device chip in the above-mentioned network camera to generate a scene image motion detection value set sequence; based on the above-mentioned scene image motion detection value set sequence, performs motion detection on each segmented scene image in the segmented scene image set sequence. The segmented scene images in the set are associated with each other to generate an image connection area, and an image connection area set is obtained, wherein the image connection area is the largest associated area in the segmented scene image; for each image connection area in the above image connection area set, the following control steps are performed: in response to determining that the area ratio of the above image connection area to the area of ​​the corresponding scene image is greater than or equal to a preset area ratio threshold, the above image connection area is determined to be the area where the moving object is located; the coordinates of the area center point of the above image connection area are determined, and the coordinates of the image center point of the corresponding scene image are determined; based on the coordinates of the area center point and the coordinates of the image center point, the pan / tilt of the above network camera is controlled to rotate to track the movement of the object.

[0087] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0088] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0089] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The units described may also be provided in a processor, for example, may be described as: a processor including an acquisition unit, an image segmentation unit, a motion detection unit, an image association unit, and a pan / tilt control unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the acquisition unit may also be described as a "unit for acquiring scene videos captured by a network camera".

[0090] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0091] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A network camera pan / tilt control method for object movement tracking, include: Acquire a scene video captured by a network camera, wherein the scene video includes a scene image sequence of continuous frames; Performing image segmentation on each scene image in the scene image sequence in the scene video to generate a sequence of segmented scene image sets; Performing motion detection on each segmented scene image in the segmented scene image set sequence by an image processing module deployed in a device chip in the network camera to generate a scene image motion detection value set sequence; Based on the scene image motion detection value set sequence, each segmented scene image in each segmented scene image set in the segmented scene image set sequence is associated with each other to generate an image connected area, thereby obtaining an image connected area set, wherein the image connected area is the largest associated area in the segmented scene image; For each image connected area in the image connected area set, the following control steps are performed: In response to determining that the ratio of the area of ​​the image connection area to the area of ​​the corresponding scene image is greater than or equal to a preset area ratio threshold, determining that the image connection area is an area where a moving object is located; Determine the coordinates of the area center point of the image connection area, and determine the coordinates of the image center point of the corresponding scene image; Based on the coordinates of the center point of the region and the coordinates of the center point of the image, the pan / tilt of the network camera is controlled to rotate to track the movement of the object.

2. The method according to claim 1, in, The method further comprises: Determine the object moving direction vector based on the positional relationship between the coordinates of the center point of the region and the coordinates of the center point of the image; Rendering the object moving direction vector to a corresponding position in each scene image in the scene image sequence, and marking an object detection frame in each scene image to obtain a rendered scene image sequence; The rendered scene image sequence is sent to a display terminal for display.

3. The method according to claim 1, in, The method further comprises: In response to determining that a ratio of an area of ​​the image connection region to an area of ​​the corresponding scene image is less than a preset area ratio threshold, it is determined that no moving object is detected in the scene video.

4. The method according to claim 1, in, The performing image segmentation on each scene image in the scene image sequence in the scene video to generate a segmented scene image set sequence includes: For each scene image in the scene image sequence, the scene image is segmented according to preset segmentation data to generate a segmented scene image set, wherein the sizes of the segmented scene images in the segmented scene image set are the same.

5. The method according to claim 1, in, The performing motion detection on each segmented scene image in the segmented scene image set sequence to generate a scene image motion detection value set sequence includes: Performing pixel comparison on the segmented scene images at corresponding positions of two adjacent segmented scene image sets in the segmented scene image set sequence to generate a total pixel difference value, thereby obtaining a total pixel difference value set; A total pixel difference set between each segmented scene image set in the segmented scene image set sequence and an adjacent previous segmented scene image set is determined as a scene image motion detection value set to obtain a scene image motion detection value set sequence.

6. The method according to claim 1, in, The step of performing association processing on each segmented scene image in each segmented scene image set in the segmented scene image set sequence based on the scene image motion detection value set sequence to generate an image connected area includes: Selecting segmented scene images whose corresponding scene image motion detection values ​​are greater than a preset detection threshold from the segmented scene image set to generate a target scene image group; The target scene images in the target scene image group are regionally connected to generate an image-connected region.

7. The method according to claim 1, in, The method of controlling the pan / tilt of the network camera to rotate based on the coordinates of the center point of the region and the coordinates of the center point of the image to track the movement of the object includes: Determine the area ratio of the image connection area in the corresponding scene image; Determine the pixel distance value between the coordinates of the center point of the region and the coordinates of the center point of the image; In response to determining that the horizontal coordinate value of the midpoint coordinate of the region is less than the horizontal coordinate value of the center point coordinate of the image, the region proportion is greater than a preset region proportion threshold, and the pixel distance value is greater than a preset distance threshold, controlling the pan / tilt of the network camera to rotate leftward, wherein when the horizontal coordinate value of the midpoint coordinate of the region is equal to the horizontal coordinate value of the center point coordinate of the image, no left / right rotation is performed; In response to determining that the horizontal coordinate value of the midpoint coordinates of the region is greater than the horizontal coordinate value of the center coordinates of the image, the region proportion is greater than a preset region proportion threshold, and the pixel distance value is greater than a preset distance threshold, controlling the pan / tilt of the network camera to rotate rightward; In response to determining that the ordinate value of the midpoint coordinates of the region is less than the ordinate value of the coordinates of the center point of the image, and the pixel distance value is greater than a preset distance threshold, controlling the pan / tilt of the network camera to rotate downward, wherein when the ordinate value of the midpoint coordinates of the region is less than the ordinate value of the coordinates of the center point of the image, no upward or downward rotation is performed; In response to determining that the ordinate value of the midpoint coordinates of the region is greater than the ordinate value of the center point coordinates of the image, and the pixel distance value is greater than a preset distance threshold, the pan / tilt of the network camera is controlled to rotate upward, wherein the distances between the horizontal and vertical coordinate values ​​of the center point coordinates of the region and the center point coordinates of the image are different, and the angles of controlling the pan / tilt to rotate in different directions are different.

8. A network camera pan / tilt control device for object movement tracking, include: An acquisition unit is configured to acquire a scene video captured by a network camera, wherein the scene video includes a scene image sequence of continuous frames; An image segmentation unit is configured to perform image segmentation on each scene image in the scene image sequence in the scene video to generate a segmented scene image set sequence; a motion detection unit configured to perform motion detection on each segmented scene image in the segmented scene image set sequence through an image processing module deployed in a device chip in the network camera, so as to generate a scene image motion detection value set sequence; an image associating unit configured to, based on the scene image motion detection value set sequence, perform an associating process on each segmented scene image in each segmented scene image set in the segmented scene image set sequence to generate an image connected region, and obtain an image connected region set, wherein the image connected region is the largest associated region in the segmented scene image; The pan / tilt control unit is configured to perform the following control steps for each image communication area in the image communication area set: In response to determining that the ratio of the area of ​​the image connection area to the area of ​​the corresponding scene image is greater than or equal to a preset area ratio threshold, determining that the image connection area is an area where a moving object is located; Determine the coordinates of the area center point of the image connection area, and determine the coordinates of the image center point of the corresponding scene image; Based on the coordinates of the center point of the region and the coordinates of the center point of the image, the pan / tilt of the network camera is controlled to rotate to track the movement of the object.

9. An electronic device, include: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, in, When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images