Target identification method and device

Abstract

The present disclosure relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning and the like, and provides a target identification method and device. The specific implementation comprises: obtaining a video stream collected by a rotatable camera device; determining a preset position of the camera device on the basis of a target in an image frame corresponding to the video stream; determining a still image frame and a fixed still object in the still image frame on the basis of the preset position and the video stream; and determining a position change result of the target in the video stream on the basis of the still image frame and the fixed still object. The implementation improves the accuracy of position change identification of targets.

Description

Target recognition method and device

[0001] This patent application claims priority to the Chinese patent application filed on November 14, 2023, with application number 202311515037.3 and invention name “Target Identification Method and Device”, the full text of which is incorporated by reference into this application. Technical Field

[0002] The present disclosure relates to the field of artificial intelligence technology, specifically to technical fields such as computer vision and deep learning, and more particularly to a target recognition method and device, electronic equipment, computer-readable medium, and computer program product. Background Art

[0003] As a type of camera, a dome camera can automatically adjust its angle and direction and has a preset position function. During the actual landing process, in the application scenario of the dome camera, the target may appear to "pseudo-disappear" due to the rotation of the dome camera.

[0004] Since the movement of the ball camera and the target is relative, in order to address the above-mentioned "pseudo-disappearance" phenomenon, the existing technology may mistakenly judge that the target has moved when the ball camera rotates.

[0005] Summary of the Invention

[0006] Provided are a target recognition method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0007] According to a first aspect, a target recognition method is provided, which includes: obtaining a video stream captured by a rotatable camera device; determining a preset position of the camera device based on a target in an image frame corresponding to the video stream; determining a still image frame and a fixed still object in the still image frame based on the preset position and the video stream; and determining a position change result of the target in the video stream based on the still image frame and the fixed still object.

[0008] According to a second aspect, a target recognition device is provided, which includes: an acquisition unit configured to acquire a video stream captured by a rotatable camera device; a position determination unit configured to determine a preset position of the camera device based on a target in an image frame corresponding to the video stream; an image determination unit configured to determine a still image frame and a fixed still object in the still image frame based on the preset position and the video stream; and a result determination unit configured to determine a position change result of the target in the video stream based on the still image frame and the fixed still object.

[0009] According to a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in any implementation manner of the first aspect.

[0010] According to a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause a computer to execute the method as described in any implementation of the first aspect.

[0011] According to a fifth aspect, a computer program product is provided, comprising a computer program, which implements the method described in any implementation manner of the first aspect when executed by a processor.

[0012] The target recognition method and device provided by the embodiments of the present disclosure first obtain a video stream captured by a rotatable camera device; secondly, determine the preset position of the camera device based on the target in the image frame corresponding to the video stream; thirdly, determine the still image frame and the fixed still object in the still image frame based on the preset position and the video stream; and finally, determine the position change of the target in the video stream based on the still image frame and the fixed still object. Thus, based on the preset positioning of the camera device, the target in the video stream is determined by the relative position relationship between the fixed still object and the target, which can effectively identify the state of the target and improve the reliability of the target position change recognition.

[0013] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0015] FIG1 is a flow chart of an embodiment of a target recognition method according to the present disclosure;

[0016] FIG2 is a schematic structural diagram of an embodiment of a target recognition device according to the present disclosure;

[0017] FIG3 is a block diagram of an electronic device for implementing the target recognition method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0019] In this embodiment, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0020] With the advent of the big data era, AI (Artificial Intelligence) is gradually emerging. Using cameras as AI's eyes, we can perform intelligent event detection based on video content. Object motion detection can be applied in a variety of scenarios, such as missing public buildings (such as manhole covers and stone piers) and the movement of museum artifacts.

[0021] The present disclosure provides a target recognition method that can identify the position changes of a target in a video stream provided by a rotating camera device, thereby improving the accuracy of target positioning. FIG1 shows a process 100 according to an embodiment of the target recognition method of the present disclosure. The target recognition method includes the following steps:

[0022] Step 101: Acquire a video stream captured by a rotatable camera device.

[0023] In this embodiment, the rotatable camera device is a camera device with a cruise function. For example, the camera device is a dome camera. The cruise function of the dome camera refers to the dome camera's timed and fixed-point rotation function. In order to realize the cruise function, it is necessary to first set a preset position for the dome camera. After the cruise function of the dome camera is turned on, the dome camera will cruise at the preset position according to the preset time period or time point, realizing timed rotation between each preset position.

[0024] In this embodiment, the preset positions are multiple, pre-set, rotatable positions for the camera device. Each of these preset positions is distributed relative to the camera device's rotation axis. When the camera device rotates to a preset position, it captures an image at that preset position and generates a video stream. The execution entity obtains the video stream sent by the camera device and, based on the information in the video stream, determines the current preset position of the camera device.

[0025] Step 102: Determine a preset position of the camera device based on the target in the image frame corresponding to the video stream.

[0026] In this embodiment, the target is the subject to be monitored by the target recognition method of the present disclosure. By analyzing the target in the image frame of the video stream, it can be determined whether the camera device is rotating or remains in a fixed position.

[0027] In this embodiment, the video stream can be converted into multiple image frames through an image conversion tool, and the targets in the image frames can be identified. For changes in targets in image frames at different times, it can be determined that the position of the camera device has not changed; by looking up the time corresponding to the image frame in which the position of the camera device has not changed and searching the corresponding time and preset position correspondence table for the moment, the preset position of the camera device can be obtained.

[0028] In this embodiment, the preset position is a position pre-set for the camera device and can be rotated to. The camera device captures different images at different preset positions. Therefore, the video stream reflects the image information of the camera device at different preset positions.

[0029] Step 103: Determine a still image frame and a fixed still object in the still image frame based on the preset position and the video stream.

[0030] In this embodiment, when the camera device is at a preset position, an image frame corresponding to the preset position can be obtained through the video stream corresponding to the preset position; the fixed still object in the image frame corresponding to the preset position is identified to obtain a still image frame having the same fixed still object.

[0031] In this embodiment, the above-mentioned step 103 includes: determining the rotation moment of the camera device based on the preset position; extracting the image frame corresponding to the rotation moment from the video stream based on the rotation moment; identifying the fixed still object in the image frame corresponding to the rotation moment, and using the image frame with the fixed still object as the still object image frame.

[0032] Step 104 : determining a position change result of the target in the video stream based on the still image frame and the fixed still object.

[0033] In this embodiment, the above-mentioned step 104 includes: converting the video stream into multiple image frames through an image conversion tool; selecting a still image frame and an image frame to be tested after the still image frame in the image frame according to the time sequence of the video stream, performing fixed still object recognition on the image frame to be tested, and in response to the presence of a fixed still object in the image frame to be tested, performing target detection on the image to be tested having the fixed still object; in response to the presence of a target in the image to be tested having the fixed still object, determining that the position of the target in the image frame to be tested has not changed.

[0034] The above step 104 may further include: if the image to be measured having the fixed still object does not have the target, determining that the position of the target in the image frame to be measured has changed.

[0035] The target recognition method provided by the embodiments of the present disclosure first obtains a video stream captured by a rotatable camera device; secondly, determines the preset position of the camera device based on the target in the image frame corresponding to the video stream; thirdly, determines the still image frame and the fixed still object in the still image frame based on the preset position and the video stream; and finally, determines the position change of the target in the video stream based on the still image frame and the fixed still object. Thus, based on the preset positioning of the camera device, the target in the video stream is determined by the relative position relationship between the fixed still object and the target, which can effectively identify the state of the target and improve the reliability of the target position change recognition.

[0036] Dome cameras are a common type of camera used in a variety of surveillance scenarios. When used with dome cameras, many object motion detection algorithms fail due to their timed rotation to various preset positions. The disclosed object recognition method effectively addresses the object motion detection problem of dome cameras by determining the dome camera's preset positions, calibrating fixed static objects, and determining whether the current frame is at the preset position to be identified.

[0037] In some embodiments of the present disclosure, the above-mentioned determination of the preset position of the camera device based on the target in the image frame corresponding to the video stream includes: obtaining multiple image frames at different times based on the video stream; performing target detection on the image frames to obtain recognition results of the target in the image frames and the time when the target appears; and determining the preset position of the camera device based on the time of appearance and the recognition result.

[0038] In this optional implementation, a video image extraction tool can be used to extract multiple image frames at different times from the video stream. Specifically, the video image extraction tool can extract image frames from the video stream at certain intervals to obtain multiple image frames at different times.

[0039] In this optional implementation, an image recognition algorithm or a target detection model is used to obtain a detection result of whether there is a target in the image frame. When there is a target in the image frame, the time when the target appears is determined based on the time when the image frame is in the video stream; based on the time when the target appears, the time period-preset position correspondence table pre-set for the camera device is queried to determine the preset position corresponding to the time when the target appears.

[0040] The method provided in this embodiment for determining the preset position of a camera device relative to a target obtains multiple image frames at different times based on a video stream; performs target detection on the image frames to obtain recognition results of the target in the image frames and the time at which the target appears; and determines the preset position of the camera device based on the time of appearance and the recognition result, thereby improving the accuracy of obtaining the preset position.

[0041] Optionally, the preset position of the camera device relative to the target can also be determined by the calibration object and the target, wherein the calibration object is an object that can be photographed after the camera device rotates to each preset position. The above-mentioned determination of the preset position of the camera device based on the target in the image frame corresponding to the video stream includes: obtaining multiple image frames at different times based on the video stream; performing target detection on the image frame to obtain recognition results of whether the image frame contains the calibration object and the target, and determining the preset position of the camera device for the recognition results with the calibration object and the target.

[0042] In some optional implementations of this embodiment, the above-mentioned target detection on the image frame to obtain the recognition result that the image frame has the target and the time when the target appears includes: acquiring a first image frame at a first moment and a second image frame at a second moment in the image frame; using a target detection model to perform target detection on the first image frame and the second image frame to obtain a first coordinate frame of the first image frame and a second coordinate frame of the second image frame; the target detection model is used to characterize the correspondence between the image and the coordinate frame of the target in the image; calculating the intersection-and-union ratio of the first coordinate frame and the second coordinate frame; in response to the intersection-and-union ratio of the first coordinate frame and the second coordinate frame being greater than a first preset threshold, obtaining the recognition result that the image frame has the target; and determining the time when the target appears based on the first moment and the second moment.

[0043] In this optional implementation, the first moment and the second moment are moments with a certain order, for example, the first moment is earlier than the second moment. The target detection model is a model that detects targets in an image. By inputting the image into the target detection model, the target type, the target confidence, and the target coordinate box output by the target detection model can be obtained. The target coordinate box is used to frame the rectangular box of the target in the image, and the points on the rectangular box are the coordinates on the outline of the target. The target coordinate box can be used to determine the area of ​​the target in the image. The target detection model can use Faster Rcnn (Faster Regions with CNN features, a method of applying deep learning to target detection), Yolo and other network structures.

[0044] In this embodiment, the first coordinate frame is a rectangular frame used to select the target in the first image frame, and is used to reflect the area of ​​the target in the first image frame; the second coordinate frame is a rectangular frame used to select the target in the second image frame, and is used to reflect the area of ​​the target in the second image frame; the above-mentioned calculation of the intersection ratio of the first coordinate frame and the second coordinate frame includes: normalizing the first coordinate frame and the second coordinate frame to the same plane, calculating the intersection area of ​​the first coordinate frame and the second coordinate frame, calculating the union area of ​​the union of the first coordinate frame and the second coordinate frame, dividing the intersection area by the union area, and obtaining the intersection ratio of the first coordinate frame and the second coordinate frame.

[0045] Determining the target appearance time based on the first moment and the second moment includes averaging the first moment and the second moment to obtain an average time, and using the average time as the target appearance time. Optionally, determining the target appearance time based on the first moment and the second moment further includes selecting any time between the first moment and the second moment as the target appearance time.

[0046] In this optional implementation, the first preset threshold can be set according to detection requirements. For example, the first preset threshold is 0.95.

[0047] The embodiment of the present disclosure provides a method for obtaining the recognition result of a target and the appearance time of the target, obtaining a first image frame at a first moment and a second image frame at a second moment in an image frame; using a target detection model to perform target detection on the first image frame and the second image frame, in response to the intersection and union ratio of the first coordinate frame of the first image frame and the second coordinate frame of the second image frame being greater than a first preset threshold, obtaining a recognition result that the image frame has the target, and determining the appearance time of the target based on the first moment and the second moment; thereby, by extracting the first image frame and the second image frame from the image frame to detect the target in the image, the reliability of target detection is improved and the effectiveness of target recognition in the image frame is guaranteed.

[0048] In some embodiments of the present disclosure, the above-mentioned determination of the still image frame and the fixed still object in the still image frame based on the preset position and the video stream includes: determining the moment image frame corresponding to the time period based on the time period corresponding to the preset position and the video stream; performing fixed still object detection on the moment image frame to obtain the still image and the fixed still object in the still image frame.

[0049] In this embodiment, there can be multiple preset positions for the rotatable camera device, and the movement order of the camera device in the multiple preset positions can be determined based on the shooting requirements of the camera device. For example, in the first time period, the rotatable camera device is at the first preset position; in the second time period, the rotatable camera device is at the second preset position.

[0050] In this optional implementation, after determining the preset position, the time period corresponding to the preset position is determined. Specifically, the time period corresponding to the preset position can be obtained by querying the time period-preset position correspondence table stored in the camera device; since the time period includes multiple moments, and each time period of the corresponding video stream corresponds to an image frame of the time period, through the video conversion image tool, one image frame from the image frames of the time period is selected as the moment image frame.

[0051] In this optional implementation, a fixed still life is an object that remains stationary for a certain period of time (for example, one year), such as a building, a plant, or a natural scenery; a still life image frame is a moment image frame in an image that includes a fixed still life. It should be noted that the time period corresponding to the preset position includes multiple moments, and each moment may have a corresponding moment image frame. In order to determine the still life image frame, all moment image frames in the time period corresponding to the preset position may be detected at the same time, or fixed still life detection may be performed on the moment images corresponding to each moment in the time period in sequence. If a fixed still life is detected in a moment image frame, the still life image frame is determined and the fixed still life in the still life image frame is identified.

[0052] In this optional implementation, after obtaining the fixed still object, an image of the fixed still object in the still object image frame may be extracted, and the extracted image of the fixed still object may be used in determining the position change result of the target.

[0053] The method for determining still image frames and fixed still objects in still image frames provided by this optional implementation method determines the moment image frames corresponding to the time period and the video stream corresponding to the preset position; performs fixed still object detection on the moment image frames to obtain still image frames and fixed still objects in the still image frames, providing a reliable implementation method for obtaining still image frames and fixed still objects.

[0054] In another embodiment of the present disclosure, the performing of fixed still object detection on the moment image frame to obtain the still object image frame and the fixed still object in the still object image frame includes:

[0055] A pre-trained fixed still object recognition model is used to detect specific types of fixed still objects in moment image frames to obtain still object image frames and fixed still objects in still object image frames. The fixed still object recognition model is used to characterize the correspondence between image frames and specific types of fixed still objects.

[0056] In this optional implementation, the specific type of fixed still object output by the fixed still object recognition model may be multiple types of fixed still objects with different confidence levels. To this end, the confidence levels of the multiple types of fixed still objects can be sorted in ascending or descending order to obtain the fixed still object with the highest confidence level as the fixed still object in the still object image frame, or the fixed still object with a preset position having a higher confidence level can be used as the fixed still object in the still object image frame.

[0057] The still image frames and the method for obtaining fixed still objects in still image frames provided in this embodiment use a pre-trained fixed still object recognition model to identify fixed still objects of characteristic types in moment image frames, thereby improving the accuracy of obtaining still image frames and fixed still objects.

[0058] In some optional implementations of this embodiment, the above-mentioned fixed still object detection on the moment image frame to obtain the still object image frame and the fixed still object in the still object image frame includes: obtaining custom still object features; based on the custom still object features, performing fixed still object detection on the moment image frame to obtain the still object image frame and the fixed still object in the still object image frame.

[0059] In this embodiment, the custom still life feature can be a feature of a specific type of fixed still life input by the user through the user interface. By setting the custom still life feature, the flexibility of the user-defined still life feature can be increased. For example, the user can modify or delete the custom still life feature in real time through the user interface.

[0060] In this embodiment, the above-mentioned fixed still object detection is performed on the moment image frame based on the custom still object feature to obtain the still object image frame and the fixed still object in the still object image frame, including: extracting the still object feature of the moment image frame to obtain the extracted feature; comparing the custom still object feature with the extracted feature for similarity; in response to the similarity between the custom still object feature and the extracted feature being greater than a similarity threshold, determining that the moment image frame is a still object image frame; identifying the still object related to the custom still object feature in the still object image frame to obtain the fixed still object.

[0061] The method for obtaining still image frames and fixed still objects in still image frames provided by this optional implementation method performs fixed still object detection on moment image frames through customized still object features to obtain still image frames and fixed still objects in still image frames, providing another implementation method for obtaining still image frames and fixed still objects, and improving the flexibility of obtaining still image frames and fixed still objects.

[0062] In some optional implementations of the present disclosure, the above-mentioned determination of the position change result of the target in the video stream based on the still image frame and the fixed still object includes: for the image frame to be tested that is located after the still image frame in the image frame, detecting whether there is a fixed still object in the image frame to be tested; in response to the presence of a fixed still object in the image frame to be tested, detecting whether there is a target in the image frame to be tested; in response to the presence of a target in the image frame to be tested, determining that the position of the target has not changed.

[0063] In this optional implementation, after determining that the position of the target has not changed, the fact that the target position has not changed is used as a result of the position change of the target in the video stream.

[0064] In this optional implementation, a video image recognition tool can be used to obtain image frames corresponding to a video stream. Based on the temporal nature of the video stream, the image frames corresponding to the video stream also have temporal nature. After obtaining the image frames corresponding to the video stream, a still image frame is determined within the image frames corresponding to the video stream. An image frame located after (temporally delayed in) the still image frame is then determined within the image frames, and the image frame located after the still image frame is used as the image frame to be tested. It should be noted that the image frame to be tested can be an image frame within the time period corresponding to the preset position, or can be an image frame outside the time period corresponding to the preset position. By using the fixed still object and target in the image frame to be tested, it is possible to detect in real time whether the target has changed position.

[0065] In this optional implementation, the above-mentioned detection of whether there is a fixed still object in the image frame to be tested includes: using a pre-trained fixed still object recognition model to perform specific types of fixed still object detection on the image frame to be tested, and obtaining a detection result of whether there is a fixed still object in the image frame to be tested.

[0066] The method for determining the position change result of a target in a video stream provided by this optional implementation manner detects, for an image frame to be tested that follows a still object image frame in the image frame, whether the image frame to be tested contains a fixed still object; when the image frame to be tested contains a fixed still object, detects whether the image frame to be tested containing the fixed still object contains a target; and in response to the presence of a target in the image frame to be tested containing the fixed still object, determines that the position of the target has not changed.

[0067] Optionally, the above-mentioned determination of the position change result of the target in the video stream based on the still image frame and the fixed still object includes: for the image frame to be tested that is located after the still image frame in the image frame, detecting whether there is a fixed still object in the image frame to be tested; in response to the presence of a fixed still object in the image frame to be tested; detecting whether there is a target in the image frame to be tested with the fixed still object; in response to the absence of a target in the image frame to be tested with the fixed still object, determining that the position of the target has changed, and taking the changed position as the position change result of the target.

[0068] Optionally, the above-mentioned determination of the position change result of the target in the video stream based on the still image frame and the fixed still object includes: based on the video stream, obtaining the image frame of the corresponding video stream in real time; for the image frame to be tested that is located after the still image frame in the image frame; selecting the image frame of the time period corresponding to the preset position in the image frame to be tested; identifying whether there is a fixed still object in the image frame of the time period corresponding to the preset position, and obtaining the initial image frame in response to the presence of a fixed still object in the image frame of the time period corresponding to the preset position; identifying whether there is a target in the initial image frame; in response to the presence of the target in the initial image frame, determining that the position of the target has not changed; in response to the absence of the target in the initial image frame, determining that the position of the target has changed.

[0069] In some optional implementations of the present disclosure, the above-mentioned detection of whether there is a target in the image frame to be tested includes: using a target detection model to identify the image frame to be tested, obtaining a coordinate frame to be tested of the target, and the target detection model is used to characterize the correspondence between the image and the coordinate frame of the target in the image; calculating the intersection-and-union ratio of the coordinate frame to be tested and a preset target coordinate frame; in response to the intersection-and-union ratio of the coordinate frame to be tested and the target coordinate frame being less than a second preset threshold, determining that there is no target in the image frame to be tested.

[0070] In this optional implementation, the above-mentioned calculation of the intersection-union ratio of the coordinate frame to be measured and the preset target coordinate frame includes: calculating the intersection area of ​​the coordinate frame to be measured and the preset target coordinate frame; calculating the union area of ​​the coordinate frame to be measured and the target coordinate frame; and dividing the intersection area of ​​the coordinate frame to be measured and the target coordinate frame by the union area to obtain the intersection-union ratio of the coordinate frame to be measured and the target coordinate frame.

[0071] In this optional implementation, the second preset threshold can be set according to target detection requirements. Generally, the second preset threshold is smaller than the first preset threshold. For example, the second preset threshold is 0.5.

[0072] This optional implementation provides a method for detecting whether there is a target in the image frame to be tested. The image frame to be tested is identified by a target detection model to obtain a coordinate frame to be tested. The intersection-and-union ratio of the coordinate frame to be tested and the preset target coordinate frame is calculated, and the intersection-and-union ratio is used to determine whether there is a target in the image to be tested. This provides a reliable implementation method for detecting the position change of the target, ensuring the reliability of the detection of the position change of the target.

[0073] In some optional implementations of the present disclosure, the above-mentioned detecting whether there is a fixed still object in the image frame to be tested, which is located after the still object image frame in the image frame, includes: identifying the still object coordinate frame of the fixed still object in the image frame to be tested, which is located after the still object image frame in the image frame; calculating the intersection-and-union ratio of the still object coordinate frame and the preset coordinate frame; and determining that there is a fixed still object in the image frame to be tested, in response to the intersection-and-union ratio of the still object coordinate frame and the preset coordinate frame being greater than a third preset threshold.

[0074] In this optional implementation, the object detection model is modified to recognize fixed still objects as a feature type, enabling the model to obtain a still object coordinate frame for fixed still objects. Specifically, the modification includes retraining the object detection model using fixed still objects as a feature type. After the object detection model meets training completion criteria, a still object detection model is obtained that can recognize still object coordinate frames for fixed still objects in image frames. The image frame to be tested is input into the still object detection model, and the still object detection model outputs a still object coordinate frame.

[0075] In this optional implementation, the preset coordinate frame is a still object coordinate frame obtained after the still object image frame is obtained and a still object detection model is used to detect a fixed still object in the still object image frame.

[0076] In this optional implementation, the third preset threshold can be set according to the fixed still object detection requirement. For example, the third preset threshold is 0.9.

[0077] This optional implementation provides a method for detecting whether there is a fixed still object in the image frame to be tested, identifies the still object coordinate frame of the fixed still object in the image frame to be tested, calculates the intersection-and-union ratio of the still object coordinate frame and a preset coordinate frame, and determines whether there is a fixed still object in the image to be tested based on the obtained intersection-and-union ratio. This provides a reliable implementation method for identifying fixed still objects in the image frame to be tested, thereby improving the reliability of fixed still object detection.

[0078] Since the rotation of a camera device (such as a dome camera) may cause the new image and the old image to coincidentally contain a fixed still object at the same position, and in fact the fixed still object in the new image may be different from the fixed still object in the old image, for this reason, whether the image frame to be tested contains a fixed still object can be determined by similarity matching of the fixed still object. In another optional implementation of the present disclosure, before determining whether the image frame to be tested contains a fixed still object, the above-mentioned detection of whether the image frame to be tested contains a fixed still object for the image frame to be tested that is located after the still object image frame in the image frame further includes: extracting the fixed still object in the still object image frame to obtain a calibration still object image; extracting the fixed still object in the image frame to be tested to obtain a still object image to be tested; calculating the similarity value between the calibration still object image and the still object image to be tested; and determining that the image frame to be tested contains a fixed still object in response to the similarity value being greater than a similarity threshold.

[0079] In this optional implementation, the similarity value between the calibrated still life image and the still life image to be tested is calculated, and a match is considered if the similarity is greater than a similarity threshold. The similarity can be calculated using image features. Specifically, deep learning and traditional image methods can be used to extract image features, and the similarity between the image features is calculated to obtain the similarity between the calibrated still life image and the still life image to be tested.

[0080] In this optional implementation, calibrating the still life image is to identify a fixed still life in the still life image frame after obtaining the still life image frame, and extract an image obtained by the fixed still life.

[0081] This optional implementation provides a method for detecting whether a fixed still object is present in an image frame to be tested, including: identifying a still object coordinate frame of a fixed still object in an image frame to be tested that is located after a still object image frame in the image frame; calculating an intersection-and-union ratio between the still object coordinate frame and a preset coordinate frame; extracting the fixed still object from the still object image frame to obtain a calibrated still object image in response to the intersection-and-union ratio between the still object coordinate frame and the preset coordinate frame being greater than a third preset threshold; extracting the fixed still object from the image frame to be tested to obtain a still object image to be tested; calculating a similarity value between the calibrated still object image and the still object image to be tested; and determining that a fixed still object is present in the image frame to be tested in response to the similarity value being greater than a similarity threshold. This optional implementation can further improve the reliability of fixed still object detection.

[0082] Optionally, multi-frame matching can be used to avoid occlusion of the target. For example, an alarm will only be output if the target fails to match in multiple consecutive frames. If the operator fails to recall the target, a similarity matching method can be introduced to calculate the similarity between the fixed static coordinate frame cutout of the calibration frame and the corresponding cutout of the current frame. If the similarity is greater than a threshold, it is considered that the target still exists and the operator did not recall the target, resulting in a mismatch.

[0083] With further reference to FIG2 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a target recognition device. This device embodiment corresponds to the method embodiment shown in FIG1 , and the device can be specifically applied to various electronic devices.

[0084] As shown in FIG2 , the target recognition device 200 provided in this embodiment includes: an acquisition unit 201, a position determination unit 202, an image determination unit 203, and a result determination unit 204. The acquisition unit 201 can be configured to acquire a video stream captured by a rotatable camera device. The position determination unit 202 can be configured to determine the preset position of the camera device based on the target in the image frame corresponding to the video stream. The image determination unit 203 can be configured to determine the still image frame and the fixed still object in the still image frame based on the preset position and the video stream. The result determination unit 204 can be configured to determine the position change result of the target in the video stream based on the still image frame and the fixed still object.

[0085] In this embodiment, the specific processing of the acquisition unit 201, the position determination unit 202, the image determination unit 203, and the result determination unit 204 in the target recognition device 200 and the technical effects brought about by them can be referred to the relevant descriptions of step 101, step 102, step 103, and step 104 in the corresponding embodiment of Figure 1, and will not be repeated here.

[0086] In some optional implementations of this embodiment, the above-mentioned position determination unit 202 is configured to: obtain multiple image frames at different times based on the video stream; perform target detection on the image frames to obtain recognition results of the target in the image frames and the time when the target appears; and determine the preset position of the camera device based on the time of appearance and the recognition result.

[0087] In some optional implementations of this embodiment, the above-mentioned position determination unit 202 is further configured to: obtain a first image frame at a first moment and a second image frame at a second moment in the image frame; use a target detection model to perform target detection on the first image frame and the second image frame to obtain a first coordinate frame of the first image frame and a second coordinate frame of the second image frame; the target detection model is used to characterize the correspondence between the coordinate frame of the image and the target in the image; calculate the intersection-and-union ratio of the first coordinate frame and the second coordinate frame; in response to the intersection-and-union ratio of the first coordinate frame and the second coordinate frame being greater than a first preset threshold, obtain an identification result that the image frame has a target; based on the first moment and the second moment, determine the moment of appearance of the target.

[0088] In some optional implementations of the present disclosure, the above-mentioned image determination unit 203 is configured to: determine the moment image frame corresponding to the time period based on the time period and video stream corresponding to the preset position; perform fixed still object detection on the moment image frame to obtain the still object image frame and the fixed still object in the still object image frame.

[0089] In some optional implementations of the present disclosure, the above-mentioned image determination unit 203 is further configured to: use a pre-trained fixed still object recognition model to detect a specific type of fixed still object in the moment image frame, and obtain a still object image frame and a fixed still object in the still object image frame. The fixed still object recognition model is used to characterize the correspondence between the image frame and the specific type of fixed still object.

[0090] In some optional implementations of the present disclosure, the image determination unit 203 is further configured to: obtain custom still life features; perform fixed still life detection on the moment image frame based on the custom still life features to obtain a still life image frame and a fixed still life in the still life image frame.

[0091] In some optional implementations of the present disclosure, the result determination unit 204 is configured to: detect whether there is a fixed still object in the image frame to be tested that is located after the still object image frame in the image frame; in response to the presence of a fixed still object in the image frame to be tested, detect whether there is a target in the image frame to be tested; and in response to the presence of a target in the image frame to be tested, determine that the position of the target has not changed.

[0092] In some optional implementations of the present disclosure, the above-mentioned result determination unit 204 is further configured to: use a target detection model to identify the image frame to be measured to obtain the coordinate frame to be measured of the target, and the target detection model is used to characterize the correspondence between the image and the coordinate frame of the target in the image; calculate the intersection-and-union ratio of the coordinate frame to be measured and the preset target coordinate frame; in response to the intersection-and-union ratio of the coordinate frame to be measured and the target coordinate frame being less than a second preset threshold, determine that there is no target in the image frame to be measured.

[0093] In some optional implementations of the present disclosure, the result determination unit 204 is further configured to: identify, for the image frame to be tested that is located after the still image frame in the image frame, a still object coordinate frame of a fixed still object in the image frame to be tested; calculate an intersection-and-union ratio of the still object coordinate frame and a preset coordinate frame; and determine that there is a fixed still object in the image frame to be tested in response to the intersection-and-union ratio of the still object coordinate frame and the preset coordinate frame being greater than a third preset threshold.

[0094] In some optional implementations of the present disclosure, the above-mentioned result determination unit 204 is further configured to: extract the fixed still life in the still life image frame to obtain a calibrated still life image; extract the fixed still life in the image frame to be tested to obtain a still life image to be tested; calculate the similarity value between the calibrated still life image and the still life image to be tested; in response to the similarity value being greater than the similarity threshold, determine that there is a fixed still life in the image frame to be tested.

[0095] The target recognition device provided by the embodiments of the present disclosure first has an acquisition unit 201 acquire a video stream captured by a rotatable camera device; secondly, a position determination unit 202 determines a preset position of the camera device based on the target in the image frame corresponding to the video stream; thirdly, an image determination unit 203 determines a still image frame and a fixed still object in the still image frame based on the preset position and the video stream; and finally, a result determination unit 204 determines the position change result of the target in the video stream based on the still image frame and the fixed still object. Thus, based on the preset positioning of the camera device, the target in the video stream is determined by the relative position relationship between the fixed still object and the target, thereby effectively identifying the state of the target and improving the reliability of identifying the target's position change.

[0096] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0097] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0098] FIG3 shows a schematic block diagram of an example electronic device 300 that can be used to implement an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0099] As shown in Figure 3, the device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. Various programs and data required for the operation of the device 300 can also be stored in the RAM 303. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0100] Various components in device 300 are connected to I / O interface 305, including: an input unit 306, such as a keyboard, mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a magnetic disk, optical disk, etc.; and a communication unit 309, such as a network card, modem, wireless communication transceiver, etc. The communication unit 309 allows device 300 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0101] The computing unit 301 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as the target recognition method. For example, in some embodiments, the target recognition method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the target recognition method described above can be performed. Alternatively, in other embodiments, the computing unit 301 can be configured to perform the target recognition method by any other suitable means (e.g., by means of firmware).

[0102] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0103] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable target recognition device so that the program code, when executed by the processor or controller, causes the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0104] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0105] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0106] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0107] Computer systems may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises through computer programs running on the respective computers and having a client-server relationship to each other.

[0108] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0109] The above specific embodiments do not limit the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure are intended to be within the scope of protection of this disclosure.

Claims

1. A target recognition method, the method comprising: Acquire a video stream captured by a rotatable camera device; Determining a preset position of the camera device based on a target in an image frame corresponding to the video stream; Determine a still image frame and a fixed still object in the still image frame based on the preset position and the video stream; Based on the still image frame and the fixed still object, a position change result of an object in the video stream is determined.

2. The method according to claim 1, wherein: The determining the preset position of the camera device based on the target in the image frame corresponding to the video stream comprises: Based on the video stream, obtaining a plurality of image frames at different times; Performing target detection on the image frame to obtain a recognition result of the target in the image frame and an appearance time of the target; Based on the occurrence time and the recognition result, a preset position of the camera device is determined.

3. The method according to claim 2, wherein: The performing target detection on the image frame to obtain a recognition result of the target in the image frame and the appearance time of the target comprises: Acquire a first image frame at a first moment and a second image frame at a second moment in the image frames; Performing target detection on the first image frame and the second image frame using a target detection model to obtain a first coordinate frame of the first image frame and a second coordinate frame of the second image frame; the target detection model is used to characterize the correspondence between the coordinate frames of the image and the target in the image; Calculating an intersection-and-union ratio of the first coordinate frame and the second coordinate frame; In response to a value of an intersection-and-union ratio between the first coordinate frame and the second coordinate frame being greater than a first preset threshold, obtaining a recognition result that the image frame has the target; Based on the first time and the second time, an appearance time of the target is determined.

4. The method according to claim 1, wherein: The determining of a still image frame and a fixed still object in the still image frame based on the preset position and the video stream comprises: Based on the time period corresponding to the preset position and the video stream, determining a time image frame corresponding to the time period; Fixed still objects are detected on the moment image frame to obtain a still object image frame and a fixed still object in the still object image frame.

5. The method according to claim 4, wherein: The performing fixed still object detection on the moment image frame to obtain a still object image frame and a fixed still object in the still object image frame comprises: A pre-trained fixed still object recognition model is used to detect a specific type of fixed still object in the image frame at the moment to obtain a still object image frame and a fixed still object in the still object image frame. The fixed still object recognition model is used to characterize the correspondence between the image frame and the specific type of fixed still object.

6. The method according to claim 4, wherein: The performing fixed still object detection on the moment image frame to obtain a still object image frame and a fixed still object in the still object image frame comprises: Get custom still life features; Based on the custom still object feature, fixed still object detection is performed on the moment image frame to obtain a still object image frame and a fixed still object in the still object image frame.

7. The method according to claim 1, wherein: The determining the position change result of the target in the video stream based on the still image frame and the fixed still object comprises: For an image frame to be tested that is located after the still object image frame in the image frames corresponding to the video stream, detecting whether the image frame to be tested has the fixed still object; In response to the image frame to be tested having the fixed still object, detecting whether the image frame to be tested has the target; In response to the target being present in the image frame to be detected, it is determined that the position of the target does not change.

8. The method according to claim 7, wherein: The detecting whether the image frame to be detected has the target comprises: Using a target detection model to identify the image frame to be measured to obtain a coordinate frame of the target to be measured, wherein the target detection model is used to characterize the corresponding relationship between the image and the coordinate frame of the target in the image; Calculating the intersection and union ratio of the coordinate frame to be measured and the preset target coordinate frame; In response to the intersection-and-union ratio of the coordinate frame to be measured and the target coordinate frame being less than a second preset threshold, it is determined that the target is not present in the image frame to be measured.

9. The method according to claim 7, wherein: The detecting, for the image frame to be tested that is located after the still object image frame in the image frames, whether the image frame to be tested has the fixed still object comprises: For the image frame to be tested that is located after the still image frame in the image frame, identifying a still coordinate frame of a fixed still object in the image frame to be tested; Calculating the intersection-and-union ratio of the still object coordinate frame and a preset coordinate frame; In response to the intersection-and-union ratio of the still object coordinate frame and the preset coordinate frame being greater than a third preset threshold, it is determined that the image frame to be tested contains the fixed still object.

10. The method according to claim 9, wherein: Before determining that the image frame to be tested contains the fixed still object, detecting whether the image frame to be tested that is located after the still object image frame in the image frames contains the fixed still object further includes: Extracting a fixed still object in the still image frame to obtain a calibrated still image; Extracting the fixed still object in the image frame to be measured to obtain a still object image to be measured; Calculating a similarity value between the calibration still life image and the still life image to be tested; In response to the similarity value being greater than a similarity threshold, it is determined that the image frame to be detected contains the fixed still object.

11. A target recognition device, comprising: An acquisition unit, configured to acquire a video stream captured by a rotatable camera device; a position determination unit configured to determine a preset position of the camera device based on a target in an image frame corresponding to the video stream; An image determination unit, configured to determine a still image frame and a fixed still object in the still image frame based on the preset position and the video stream; The result determination unit is configured to determine the position change result of the target in the video stream based on the still image frame and the fixed still object.

12. The device according to claim 11, wherein The position determination unit is configured to: obtain a plurality of image frames at different times based on the video stream; Performing target detection on the image frame to obtain a recognition result of the target in the image frame and an appearance time of the target; Based on the occurrence time and the recognition result, a preset position of the camera device is determined.

13. The device according to claim 12, wherein: The position determination unit is further configured to: obtain a first image frame at a first moment and a second image frame at a second moment in the image frames; perform target detection on the first image frame and the second image frame using a target detection model to obtain a first coordinate frame of the first image frame and a second coordinate frame of the second image frame; The target detection model is used to characterize the correspondence between the image and the coordinate frame of the target in the image; calculating the intersection and union ratio of the first coordinate frame and the second coordinate frame; In response to the intersection-and-union ratio of the first coordinate frame and the second coordinate frame being greater than a first preset threshold, a recognition result that the image frame has the target is obtained; and based on the first moment and the second moment, an appearance moment of the target is determined.

14. The device according to claim 11, wherein: The image determination unit is configured to: determine a moment image frame corresponding to the time period based on the time period corresponding to the preset position and the video stream; perform fixed still object detection on the moment image frame to obtain a still object image frame and a fixed still object in the still object image frame.

15. The device according to claim 14, wherein: The image determination unit is further configured to: use a pre-trained fixed still object recognition model to detect a specific type of fixed still object on the image frame at the moment, and obtain a still object image frame and a fixed still object in the still object image frame, wherein the fixed still object recognition model is used to characterize the correspondence between the image frame and the specific type of fixed still object.

16. The device according to claim 14, wherein: The image determination unit is further configured to: obtain a custom still object feature; and based on the custom still object feature, perform fixed still object detection on the moment image frame to obtain a still object image frame and a fixed still object in the still object image frame.

17. The device according to claim 11, wherein: The result determination unit is configured to: detect whether the image frame to be tested that is located after the still object image frame in the image frames corresponding to the video stream contains the fixed still object; in response to the image frame to be tested containing the fixed still object, detect whether the image frame to be tested contains the target; in response to the image frame to be tested containing the target, determine that the position of the target has not changed.

18. The device according to claim 17, wherein: The result determination unit is further configured to: identify the image frame to be measured using a target detection model to obtain a coordinate frame to be measured of the target, wherein the target detection model is used to characterize the correspondence between the image and the coordinate frame of the target in the image; Calculating the intersection and union ratio of the coordinate frame to be measured and the preset target coordinate frame; In response to the intersection-and-union ratio of the coordinate frame to be measured and the target coordinate frame being less than a second preset threshold, it is determined that the target is not present in the image frame to be measured.

19. The device according to claim 17, wherein: The result determination unit is further configured to: for the image frame to be tested located after the still image frame in the image frame, identify a still coordinate frame of a fixed still object in the image frame to be tested; Calculating the intersection-and-union ratio of the still object coordinate frame and a preset coordinate frame; In response to the intersection-and-union ratio of the still object coordinate frame and the preset coordinate frame being greater than a third preset threshold, it is determined that the image frame to be tested contains the fixed still object.

20. The device according to claim 19, wherein The result determination unit is further configured to: extract the fixed still life in the still life image frame to obtain a calibration still life image; extract the fixed still life in the image frame to be tested to obtain a still life image to be tested; calculate the similarity value between the calibration still life image and the still life image to be tested; In response to the similarity value being greater than a similarity threshold, it is determined that the image frame to be detected contains the fixed still object.

21. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 10.

23. A computer program product, comprising a computer program, which, when executed by a processor, implements the method of any one of claims 1 to 10.

Images