Target object detection method and device, electronic equipment and storage medium

Abstract

The application discloses a target object detection method and device, electronic equipment and a storage medium. The method comprises the following steps: determining at least one candidate target object according to a first image matched with a first region; determining a behavior score of the at least one candidate target object, and determining a target object to be detected from the candidate target objects according to the behavior score of the candidate target objects; wherein the behavior score is used to represent the possibility of the target object carrying a target object; and if it is determined that the target object to be detected is detected in a second image matched with a second region, it is determined that the target object enters the second region. The application can realize accurate identification of the target object entering a specific region, and reduce the security risk.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, electronic device, and storage medium for detecting a target object. Background Technology

[0002] While the development of science and technology brings convenience to people's lives, it also brings some safety hazards. For example, electric bicycles and electric cars provide people with more travel options, but the charging equipment for electric bicycles and electric cars has a high power output, which can lead to overheating and short circuits during charging. If such high-power charging equipment is carried into elevators, stairwells, or other densely populated areas by people, robots, vehicles, or other targets, there is a risk of fire or explosion.

[0003] Current technologies for detecting whether high-power charging devices have been carried into a specific area by a target object typically involve installing physical sensors, such as weight sensors or magnetic sensors, in that area. However, this method suffers from low recognition rates, high false alarm rates, limited detection range, and high installation and maintenance costs. Another approach is to capture images of a specific area using imaging devices and then identify the high-power charging device within those images; however, this method is prone to false identification. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for detecting target objects, so as to achieve accurate identification of target objects entering a specific area and reduce security risks.

[0005] In a first aspect, embodiments of the present invention provide a method for detecting a target object, the method comprising:

[0006] Based on the first image that matches the first region, at least one candidate target object is determined;

[0007] Determine the behavior score of at least one candidate target object, and determine the target object to be detected among the candidate target objects based on the behavior score of the candidate target objects;

[0008] The behavior score is used to indicate the probability that the target object carries the target object.

[0009] If the target object is detected in a second image that matches the second region, then the target object is determined to have entered the second region.

[0010] Secondly, embodiments of the present invention also provide a target object detection device, the device comprising:

[0011] The candidate target object determination module is used to determine at least one candidate target object based on a first image that matches the first region;

[0012] The target object determination module is used to determine the behavior score of at least one candidate target object, and determine the target object to be detected from the candidate target objects based on the behavior score of the candidate target objects;

[0013] The behavior score is used to indicate the probability that the target object carries the target object.

[0014] The target object determination module is used to determine that the target object has entered the second region if the target object to be detected is detected in the second image that matches the second region.

[0015] Thirdly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a target object detection method as described in any of the embodiments of the present invention.

[0016] Fourthly, embodiments of the present invention also provide a storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform a target object detection method as described in any of the embodiments of the present invention.

[0017] The technical solution of this invention identifies candidate target objects in a first image matched in a first region, scores the candidate target objects for whether they are carrying a target object, and determines the target object to be detected from the candidate target objects based on the behavior score. If the target object to be detected is detected in a second image matched in a second region, it indicates that a target object exists in the second region. This solves the problems of low recognition rate, high false alarm rate, and limited detection range in existing technologies that directly identify target objects in the second region using physical sensors or image recognition. It achieves accurate identification of the behavior of target objects carrying high-power charging devices or other target objects into specific areas such as elevators, reducing safety risks.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a target object detection method provided in Embodiment 1 of the present invention;

[0021] Figure 2 This is a flowchart of a target object detection method provided in Embodiment 2 of the present invention;

[0022] Figure 3 This is a schematic diagram of the structure of a target object detection device provided in Embodiment 3 of the present invention;

[0023] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices. In the embodiments of this application, certain software, components, models, and other existing industry solutions may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solutions of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0026] The acquisition, transmission, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0027] Example 1

[0028] Figure 1 The flowchart of a target object detection method provided in Embodiment 1 of the present invention is applicable to the situation of detecting whether a target object is carried into a specific area. The method can be executed by a target object detection device, which can be implemented in hardware and / or software. The target object detection device can be configured in an electronic device or a server and used in conjunction with the imaging devices of the first area and the second area.

[0029] like Figure 1 As shown, the method includes:

[0030] S110. Based on the first image that matches the first region, determine at least one candidate target object.

[0031] In this embodiment, the terms "first" and "second" are used only to distinguish different regions or images and are not used to indicate order or other content. The first region can be a storage area, charging area, or entrance / exit area for a target object. The first image can be an image captured by a camera device matched to the first region. A camera device matched to the first region refers to a camera device whose field of view can cover the first region. The number of camera devices matched to the first region can be one or more, and this embodiment does not limit this. The first image can also be an infrared image scanned by an infrared sensor matched to the first region. An infrared sensor matched to the first region refers to an infrared sensor whose scanning range can cover the first region. Similarly, the number of infrared cameras can be one or more. The first image can also be an image generated based on radar echo signals collected by a radar sensor matched to the first region. A radar sensor matched to the first region refers to a radar sensor whose detection range can cover the first region. Similarly, the number of radar sensors can be one or more.

[0032] Candidate target objects refer to target objects appearing in the first area that may exhibit behaviors related to carrying target objects. In this embodiment, real-time target object recognition is performed on the first image. After candidate target objects are identified, they are continuously monitored based on the real-time acquired first image. The number of candidate target objects can be one or more. When there are multiple candidate target objects, subsequent behavior scores are calculated for each candidate target object separately.

[0033] In this embodiment, candidate target objects are determined by performing target object recognition algorithms or models on the first image, and the identified target objects are used as candidate target objects. Alternatively, based on the identified target objects, behavior recognition or dwell time statistics can be performed, and target objects whose behavior and / or dwell time meet preset conditions can be used as candidate target objects. The advantage of this approach is that it can exclude some target objects that are clearly unlikely to carry the target object, thereby saving computational resources and improving the efficiency of target object recognition.

[0034] In an optional embodiment, S110 may further include:

[0035] A1. Perform perimeter detection on the first region using a shooting device that matches the first region;

[0036] A2. If it is determined from the first image captured by the shooting device that the target object has entered the first area and the time spent in the first area is greater than or equal to a preset time threshold, then the target object is regarded as a candidate target object.

[0037] This embodiment uses candidate target object recognition based on images captured by an imaging device as an example for explanation. Perimeter detection refers to capturing images of a first region using imaging devices deployed around its perimeter, analyzing the captured images, identifying target objects, and detecting their movement trajectories and behavioral patterns within the first region.

[0038] Since the first area is a storage area or charging area for the target object, this embodiment determines that the target object that enters the first area and stays there for a long time is a target object that may be carrying the target object.

[0039] Specifically, the field of view of the shooting device matched with the first region is usually not exactly the same as the first region. Instead, to ensure complete coverage of the first region, the field of view is usually larger than the first region. Therefore, the location area of ​​the first region in the first image can be determined, and the target object can be detected. If the current position of the target object is determined to be within the aforementioned location area, then the target object is determined to have entered the first region. After determining that the target object has entered the first region, the dwell time of the target object in the first region is counted. If the dwell time is greater than or equal to a preset time threshold, such as 30 seconds, the target object is considered a candidate target object.

[0040] Furthermore, in this embodiment, when a target object is determined to enter the first area, behavioral pattern analysis can be performed on the target object to determine whether its behavior meets the conditions for carrying a target object. If a target object enters the first area, meets the conditions for carrying a target object, and its stay in the first area is greater than or equal to a preset time threshold, then the target object is considered a candidate target object. For example, taking an electric bicycle battery as the target object, meeting the conditions for carrying a target object could mean that the target object entered the first area while riding an electric bicycle. It is understandable that for a target object to carry a target object, it must first have contact with the target object or its carrier. Therefore, in this embodiment, by analyzing the target object's behavioral patterns, target objects that are clearly not likely to carry a target object are further excluded, thereby improving the accuracy of target object detection in the subsequent second area.

[0041] Specifically, behavioral pattern analysis of target objects within the first region can be performed by identifying and tracking motion trajectories of target objects across a series of first image frames, and extracting target object features such as shape, color, texture, and motion. The extracted target object features are then represented, for example, using vectorized representation or principal component analysis. Behavior recognition algorithms based on support vector machines, decision trees, or neural networks are then employed to perform behavior recognition and classification based on the extracted feature representations.

[0042] In another optional embodiment, the candidate target object can also be determined by scanning the first region with an infrared sensor that matches the first region to obtain a first image; if it is determined from the first image obtained by the infrared sensor that the target object has entered the first region and its stay time in the first region is greater than or equal to a preset time threshold, then the target object is regarded as a candidate target object.

[0043] This embodiment uses infrared images scanned by an infrared sensor for candidate target object identification as an example. Specifically, target objects in infrared images typically possess specific temperature characteristics. Therefore, based on the temperature values ​​corresponding to each pixel, the target object is segmented from the background. The position of the target object in the first image can be determined using methods such as feature matching, image region segmentation, and deep learning. Then, based on the position of the target object in the first image and the position of the first region in the first image, it is determined whether the target object has entered the first region. After determining that the target object has entered the first region, the dwell time is accumulated. If the dwell time is greater than or equal to a preset time threshold, the target object is designated as a candidate target object.

[0044] Furthermore, in this embodiment, behavioral pattern analysis of the target object can also be performed to determine whether the target object's behavior meets the carrying conditions of the target object. The specific process is similar to the process described above, and will not be repeated here.

[0045] In another optional embodiment, the candidate target object can also be determined by scanning the first region with a radar sensor that matches the first region to obtain a first image; if it is determined from the first image obtained by the radar sensor that the target object has entered the first region and its stay time in the first region is greater than or equal to a preset time threshold, then the target object is regarded as a candidate target object.

[0046] This embodiment uses radar images scanned by a radar sensor for candidate target object recognition as an example. Specifically, the pixel values ​​of pixels in a radar image can represent different radar echo signal intensities. Therefore, the position of the target object in the first image can be determined using threshold segmentation algorithms, edge detection algorithms, or deep learning-based algorithms. Based on the position of the target object in the first image and the position area of ​​the first region in the first image, it is determined whether the target object has entered the first region. After determining that the target object has entered the first region, the dwell time is accumulated. If the dwell time is greater than or equal to a preset time threshold, the target object is considered a candidate target object.

[0047] Furthermore, in this embodiment, behavioral pattern analysis of the target object can also be performed to determine whether the target object's behavior meets the carrying conditions of the target object. The specific process is similar to the process described above, and will not be repeated here.

[0048] In this embodiment, by identifying candidate target objects based on the first image matching the first region, the behavior of the target object carrying the target object into the second region can be tracked and recorded from the source, expanding the detection dimensions. This means that the computational load in the entire process of determining whether the target object has entered the second region is mainly concentrated on the first image matching the first region. Subsequently, when the target object is detected in the second image of the second region, it can be quickly determined that the target object has entered the second region, improving detection speed. Furthermore, by filtering target objects within the first region, target objects that are clearly unlikely to carry the target object are excluded, thereby improving the accuracy of subsequent target object detection in the second region.

[0049] Furthermore, this embodiment can record the candidate target object's characteristic information, dwell time, movement trajectory, and characteristic information of the objects it carries after determining the candidate target object, and establish a candidate target object file. The advantages of this setup are twofold: firstly, it facilitates subsequent behavior scoring calculations; secondly, it facilitates tracing the source and provides data support when a target object is subsequently determined to exist in the second area.

[0050] S120. Determine the behavior score of at least one candidate target object, and determine the target object to be detected among the candidate target objects based on the behavior score of the candidate target object.

[0051] The behavior score is used to represent the probability that a target object is carrying a target object. In this embodiment, the probability that a candidate target object is carrying a target object is determined by calculating a behavior score for the candidate target object; the higher the behavior score, the greater the probability that the target object is carrying the target object.

[0052] In this embodiment, the probability of a candidate target object carrying a target object is quantitatively represented by behavioral scoring. The candidate target objects carrying the target object are then filtered out using behavioral scoring. Thus, when the target object is detected in the second region, it can be determined that a target object exists in the second region, thereby improving the accuracy of target object detection in the second region.

[0053] In an optional embodiment, determining the behavior score of at least one candidate target object can be achieved by using a pre-trained behavior pattern recognition model to determine the confidence level of the candidate target object performing a preset behavior, and then using this confidence level as the behavior score of the candidate target object. The behavior pattern recognition model can be obtained by training a deep learning model on sample images labeled with behavior patterns. The preset behavior may include actions such as picking up the target object or carrying the target object by hand. Taking an electric bicycle battery as an example, the preset behavior may include picking up the battery from the battery storage location of the electric bicycle or carrying the battery by hand.

[0054] In this embodiment, each first image containing a candidate target object is input into a behavior pattern recognition model to obtain the confidence level of the candidate target object performing a preset behavior, as output by the behavior pattern recognition model. Furthermore, if it is determined that the candidate target object performs at least two preset behaviors, the highest confidence level can be used as the behavior score of the candidate target object.

[0055] It is understandable that when a candidate target object performs an action related to carrying the target object, there is a high probability that it will carry the target object out of the first area. Therefore, in this embodiment, the confidence level of the candidate target object performing the preset action is used as the action score, which can quantify the probability that the candidate target object will carry the target object out of the first area, thereby improving the accuracy of target object detection in the subsequent second area.

[0056] In another optional embodiment, further determining the behavior score of at least one candidate target object may include: determining the behavior score of the candidate target object based on at least one of the following: carrying object score, dwell time score, and action posture score; wherein the carrying object score is determined based on the object characteristics of the object carried by the candidate target object, the dwell time score is determined based on the dwell time of the candidate target object in the first area, and the action posture score is determined based on the action posture of the candidate target object in the first area.

[0057] This embodiment provides a method for determining the behavior score of candidate target objects based on multiple dimensions. It is understood that if the characteristics of the object carried by the candidate target object are similar to the target object, it should have a higher carried object score, thereby increasing the behavior score. Since the candidate target object needs time to carry the target object out of the first area, a longer stay time in the first area should result in a higher stay time score, thus increasing the behavior score. Furthermore, if the candidate target object exhibits actions such as picking up the target object, it should have a higher action posture score, thereby increasing the behavior score. This embodiment can determine the behavior score of candidate target objects based on one or more of the following factors: carried object score, stay time score, and action posture score.

[0058] Specifically, the process of determining the carrying object score can include: determining the object features of the object carried by the candidate target object, and determining the carrying object score that matches the object features. Edge detection algorithms (such as Canny edge detection) can be used to extract the edges of the candidate target object in the first image. Taking the human body as an example, the contour shapes of the human hand and the suspected carrying object are analyzed. For example, if the contour of the human hand shows a shape of holding an object, or if there is a clear object contour connected to the hand, it indicates that the candidate target object is carrying an object. If no carrying object is detected, the carrying object score can be set to 0.

[0059] After identifying the objects carried by the candidate target object, the contour features, color features, and texture features of the carried object region are extracted. Based on the size of the carried object region and the extracted features, the similarity is compared with the size and feature information of the pre-determined target object to obtain the similarity score between the carried object and the target object. This similarity score can be directly used as the carried object score of the candidate target object; alternatively, different similarity intervals can be set, with different intervals corresponding to different carried object scores, and the carried object score corresponding to the interval of similarity can be used as the carried object score of the candidate target object; alternatively, a mathematical function can be set to determine the relationship between similarity and the carried object score of the candidate target object, as long as the carried object score increases with increasing similarity. This embodiment does not limit the specific method for determining the carried object score based on similarity.

[0060] Furthermore, if an object carried by a candidate target object is detected when it enters the first region, then the object carried by the candidate target object when leaving the first region is compared with the object carried when entering the first region. If the difference between the object carried by the candidate target object when leaving the first region and the object carried when entering the first region is small, the carrying object score can be set to a small value or even 0. If the area of ​​the object carried by the candidate target object when leaving the first region is significantly larger than the area of ​​the object carried when entering the first region—for example, the ratio of the area of ​​the area of ​​the object carried when leaving the first region to the area of ​​the object carried when entering the first region is greater than or equal to a preset ratio threshold, such as 1.2—then the increased area of ​​the area of ​​the object carried by the candidate target object when leaving the first region is determined. Using the method described in the above embodiments, the size of the increased area and the extracted features are compared with the size and feature information of the predetermined target object to calculate the similarity score, which is used as the carrying object score of the candidate target object.

[0061] Specifically, the process of determining the dwell time score may include: determining the dwell time of the candidate target object within the first region, and determining a dwell time score that matches the dwell time. The dwell time can be determined as the time difference between the moment the candidate target object enters the first region and the moment it leaves the first region. The process of determining entry into the first region has been described in the above embodiments. The process of determining exit from the first region is similar to that of determining entry, and both can be determined by the current position of the candidate target object in the current first image and the position area of ​​the first region in the first image. This embodiment will not repeat the details here.

[0062] Similarly, different time intervals can be set, with different intervals corresponding to different dwell time scores. The dwell time score corresponding to the time interval in which the dwell time falls can be used as the dwell time score of the candidate target object. Alternatively, a mathematical function can be set to determine the relationship between dwell time and the dwell time score of the candidate target object. As long as the dwell time increases, the dwell time score will also increase accordingly. This embodiment does not limit the specific method of determining the score of the carried object based on similarity.

[0063] Specifically, the process of determining the action posture score may include: determining the action posture of the candidate target object within a first region, and determining the action posture score that matches the action posture. The behavior of the candidate target object can be identified using a behavior pattern recognition model, the specific process of which has been described in the above embodiments and will not be repeated here. Alternatively, a human posture estimation algorithm can be used to determine the hand joint positions of the candidate target object. If the position and posture of the hand joints match the characteristics of grasping an object, such as bent fingers or palm contact with the object, then it is determined that there is an action posture for grasping an object.

[0064] After determining that a candidate target object exhibits a preset behavior or a gesture of picking up an object, a high value, such as 0.9 or even 1, can be set for the candidate target object's gesture score. Alternatively, the gesture score can be determined based on the confidence level of the candidate target object performing the preset behavior. For example, if the confidence level of the candidate target object picking up an object is 0.7, the gesture score can be set to 0.7.

[0065] Specifically, if the candidate target object's behavior score is determined based on any one of the following: the object-carrying score, the dwell time score, or the action / posture score, then that score can be directly used as the candidate target object's behavior score. If the candidate target object's behavior score is determined based on at least two of these factors, taking the determination of the candidate target object's behavior score based on the object-carrying score, dwell time score, and action / posture score as an example, weights can be assigned to each score. The weights of each score can be the same or different. The weighted sum of the scores yields the final candidate target object's behavior score.

[0066] Furthermore, determining the target object to be detected from the candidate target objects based on the behavior scores of the candidate target objects may also include: if the behavior score of the candidate target object is determined to be greater than or equal to a preset score threshold, then the candidate target object is taken as the target object to be detected.

[0067] In this embodiment, candidate target objects with behavior scores greater than or equal to a preset score threshold are identified as target objects carrying target objects. Subsequently, target object recognition can be performed through images matched in the second region. As long as the target object is recognized, it can be identified as a target object entering the second region.

[0068] S130. If the target object to be detected is detected in the second image that matches the second region, then the target object is determined to have entered the second region.

[0069] The second region is an area where the target object is unsuitable or cannot enter; if the target object enters the second region, a safety hazard will exist. The second image can be an image captured by a camera device matched to the second region. A camera device matched to the second region refers to a camera whose field of view can cover the second region. The number of cameras matched to the second region can be one or more; this embodiment does not limit this. The second image can also be an infrared image obtained by scanning with an infrared sensor matched to the second region. An infrared sensor matched to the second region refers to an infrared sensor whose scanning range can cover the second region. Similarly, the number of infrared cameras can be one or more. The second image can also be an image generated based on radar echo signals collected by a radar sensor matched to the second region. A radar sensor matched to the second region refers to a radar whose detection range can cover the second region. Similarly, the number of radar sensors can be one or more.

[0070] The specific process of detecting the target object in the second image is the same as the specific process of identifying the target object in the first image, and will not be repeated in this embodiment. It should be noted that in this embodiment, as long as the target object is detected in the second image, it is considered that the target object has entered the second region, and there is no need to judge the behavior pattern, dwell time, etc. of the target object.

[0071] In one applicable scenario, the target object can refer to high-power charging equipment such as electric vehicle batteries or EV batteries. Correspondingly, the target object is a face or human body, the first area can be an area where electric vehicles or EVs are parked or charging, and the second area can be a stairwell or elevator. It is understandable that if high-power charging equipment such as electric vehicle batteries or EV batteries is carried into enclosed spaces such as stairwells or elevator shafts, there is a high safety risk. Therefore, this embodiment detects whether high-power charging equipment such as electric vehicle batteries or EV batteries enters stairwells or elevators.

[0072] In another example of an applicable scenario, the target object can also refer to flammable and explosive materials. Correspondingly, the target object could be a face, a human body, or a robot, etc. The first area could be an entrance, etc., and the second area could be a production workshop, station, etc. It is understandable that if flammable and explosive materials are brought into production workshops, stations, etc., there is a high safety risk. Therefore, this embodiment can detect whether flammable and explosive materials have entered production workshops, stations, etc.

[0073] In another example of an applicable scenario, the target object can also refer to a camera or other photographing device. Correspondingly, the target object is a face, a human body, etc., the first area can be an entrance, etc., and the second area can be a meeting room, etc. Understandably, some meetings have confidentiality requirements and prohibit photographing devices from entering the meeting room. Therefore, this embodiment can detect whether a photographing device has entered the meeting room, etc.

[0074] Furthermore, after confirming that the target object has entered the second area, notifications can be sent via SMS, voice prompts, or a user interface. Simultaneously, data such as the target object's characteristic information, movement trajectory, the trigger time and location of its entry into the second area, and a second image of the target object can be saved and displayed through the user interface for verification of the target object's entry into the second area.

[0075] In this embodiment, the detection of target objects within the second region is transformed into the detection of whether a target object carries another target object into the second region. On one hand, this allows for lead time for target object detection, enabling tracking and recording of target objects from the source, thus improving the speed of target object detection. On the other hand, it expands the dimensions of target object detection, not relying solely on direct detection, but rather on the behavior of a target object carrying another target object. This avoids situations where the target object itself is difficult to detect due to its size, color, shape, or occlusion. This improves the accuracy of detecting target objects entering the second region and significantly reduces false positives and false negatives.

[0076] The technical solution of this invention identifies candidate target objects in a first image matched in a first region, scores the candidate target objects for whether they carry a target object, and determines the target object to be detected from the candidate target objects based on the behavior score. If the target object to be detected is detected in a second image matched in a second region, it indicates that a target object exists in the second region. This solves the problems of low recognition rate, high false alarm rate, and limited detection range in existing technologies that directly identify target objects in the second region using physical sensors or image recognition. It achieves accurate identification of target objects entering a specific region, reducing security risks.

[0077] Example 2

[0078] Figure 2 This is a flowchart of a target object detection method provided in Embodiment 2 of the present invention. Based on the above embodiments, the present invention adds a process of trajectory tracking and motion trend prediction of the target object.

[0079] like Figure 2 As shown, the method includes:

[0080] S210. Based on the first image that matches the first region, determine at least one candidate target object.

[0081] S220. Determine the behavior score of at least one candidate target object, and determine the target object to be detected among the candidate target objects based on the behavior score of the candidate target object.

[0082] The process of determining candidate target objects, calculating the behavior scores of candidate target objects, and selecting the target object to be detected from the candidate target objects have been described in the above embodiments, and will not be repeated here.

[0083] S230, determine a first image that matches the first region, a second image that matches the second region, and a third image that matches the path between the first region and the second region.

[0084] The path between the first and second regions can be determined based on a map. The third image matching the path between the first and second regions refers to an image captured by a camera that matches the path (or an image scanned by an infrared sensor or radar sensor; in this embodiment, an image captured by a camera is used as an example to illustrate the subsequent process). There can be one or more camera devices matching the path; therefore, the third image can be an image captured by one or more camera devices. It should be noted that in this embodiment, "third," like "first" and "second," is used for distinction and not for restricting the order.

[0085] In this embodiment, the target object to be detected is continuously tracked and detected through the coordinated detection of multiple shooting devices in the first region, the second region, and the path between the first region and the second region.

[0086] S240. Determine the motion trajectory of the target object to be detected based on at least one of the first image, the second image, and the third image.

[0087] Understandably, when a target object leaves the first area and enters the second area, it is often possible for multiple cameras to capture the target object. Therefore, it is necessary to link and fuse the images from multiple cameras to obtain the motion trajectory of the target object.

[0088] Because different shooting devices have different positions and viewpoints, the position of the same target object in the images they capture is relative to its respective camera coordinate system. Therefore, it is necessary to transform the position of the same target object in images captured by different shooting devices to the same spatial coordinate system in order to perform data fusion and target matching across multiple shooting devices. Specifically, the mapping relationship between the position in the images captured by each shooting device and the position in the spatial coordinate system is determined by using the intrinsic parameters (such as focal length, pixel size, etc.) and extrinsic parameters (such as camera position information, rotation angle, etc.) of each shooting device. Based on the position of the target object in at least one of the first, second, and third images, and the above mapping relationship, the spatial position of the target object in the spatial coordinate system is determined. Based on the spatial position of the target object at different times, the motion trajectory of the target object is determined.

[0089] In an optional embodiment, similarity can be calculated to determine whether target objects captured by different imaging devices are the same target object to be detected. Specifically, after determining the target object to be detected based on the first image, if the target object is detected in the second and / or third image, the similarity between the target object and the target object to be detected is calculated. If the similarity is greater than or equal to a preset similarity threshold, the target object detected in the second and / or third image is considered the target object to be detected. It should be noted that the target object to be detected can appear simultaneously in any two or more of the first, second, and third images, or it can appear first in the first image and then in the second and / or third image. This embodiment does not impose any restrictions on this.

[0090] In another optional embodiment, after determining the target object to be detected based on the first image, the position of the target object in the first image can be converted to a spatial coordinate system. If the target object is detected in the second image and / or the third image, the position of the target object can also be converted to a spatial coordinate system. If the distance between the spatial position of the target object and the spatial position of the target object to be detected is less than or equal to a preset distance threshold, the target object is determined to be the target object to be detected.

[0091] In this embodiment, by transforming position coordinates and matching target objects in different images, the spatial position information of the target object to be detected at different times can be obtained, thereby obtaining the motion trajectory of the target object to be detected. Specifically, motion estimation algorithms such as Kalman filtering and particle filtering algorithms can be used to calculate the motion trajectory of the target object to be detected. Furthermore, the motion trajectory can be smoothed and outliers removed using sliding window methods or curve fitting methods.

[0092] In this embodiment, trajectory tracking of the target object to be detected has two advantages. First, by judging the movement trend of the trajectory, it can provide advance warning of the target object entering the second region before it actually enters, thus improving the efficiency of target object detection. Second, it can exclude target objects that have no intention of entering the second region, thereby saving computational resources.

[0093] S250. Determine whether the movement trend of the target object to be detected, based on the movement trajectory, is to enter the second region. If yes, execute S260; otherwise, execute S270.

[0094] The motion trend can be predicted using a pre-trained predictive model. Specifically, a predictive model is obtained by training a linear regression model, time series model, support vector machine model, or deep learning model using historical motion trajectory data. The motion trajectory of the target object is then input into the predictive model, which predicts the motion trend based on the target object's trajectory, current position, current velocity, and current acceleration.

[0095] S260, Provide a warning that the target object is entering the second area.

[0096] In this embodiment, if the target object to be detected has a tendency to move into the second area, it is considered that it has a tendency to carry the target object into the second area, and a warning can be given in advance that the target object is entering the second area.

[0097] Furthermore, the warning prompts can be displayed through a user interface or through voice prompts from sensors deployed in or near the second area.

[0098] In this embodiment, by providing a warning when the target object to be detected shows a tendency to enter the second area, the target object can be prevented from entering the second area at the source, thereby minimizing the safety risk.

[0099] S270. Determine whether the duration of the motion trajectory is greater than or equal to a preset duration threshold. If yes, execute S280; otherwise, execute S290.

[0100] Understandably, after a period of continuous monitoring of the target object, if the target object still does not show a trend of entering the second area, continuous monitoring will cease, and the information corresponding to the target object, including the target object's characteristic information, dwell time in each area, movement trajectory, and characteristic information of the objects it carries, will be deleted. If a file for the target object has been created, the file will be deleted.

[0101] S280. Delete the target object to be detected.

[0102] S290. Determine whether the target object to be detected is detected in the second image that matches the second region. If yes, execute S2100; otherwise, return to execute S240.

[0103] In this embodiment, if a target object is detected in the second image matching the second region, it is considered that the target object intends to enter the second region or has already entered the second region. At this point, the target object is confirmed to have entered the second region, and a notification is given that the target object has entered the second region. Similarly, the notification can be given through voice prompts, SMS notifications, or a user interface display.

[0104] Furthermore, based on the position of the target object in the second image and the position of the second region in the second image, it can be determined whether the target object has entered the second region. Only after confirming that the target object has entered the second region should a notification be issued. This setup avoids false alarms about the target object entering the second region, improving the accuracy of target object detection.

[0105] S2100, Determine that the target object has entered the second region.

[0106] The technical solution of this embodiment identifies candidate target objects in the first image of the first region, scores their behavior, and selects the candidate target objects with higher behavior scores as the target objects to be detected for subsequent tracking and intent judgment. Based on the first image of the first region, the third image of the path from the first region to the second region, and the second image of the second region, the motion trajectory of the target object to be detected is determined. When the motion trajectory of the target object to be detected shows a tendency to enter the second region, a warning is given that the target object is entering the second region. When the target object to be detected is detected entering the second region in the second image of the second region, it is determined that the target object has entered the second region, and a warning is given that the target object has entered the second region. Compared with traditional methods of detecting target objects in the second region, the technical solution of this embodiment not only allows for advance planning, ensuring the timeliness of target object detection, but also expands the detection dimensions, avoiding missed or false alarms caused by the difficulty in detecting the target object or its occlusion, thus improving the accuracy of target object detection. Furthermore, this embodiment tracks and records the entire process of the target object carrying the target object into the second region from the source, facilitating subsequent behavioral analysis and verification of the target object entering the second region.

[0107] Example 3

[0108] Figure 3 This is a schematic diagram of the structure of a target object detection device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes:

[0109] The candidate target object determination module 310 is used to determine at least one candidate target object based on a first image that matches the first region;

[0110] The target object determination module 320 is used to determine the behavior score of at least one candidate target object, and determine the target object to be detected among the candidate target objects based on the behavior score of the candidate target object;

[0111] The behavior score is used to indicate the probability that the target object carries the target object.

[0112] The target object determination module 330 is used to determine that the target object has entered the second region if the target object to be detected is detected in the second image that matches the second region.

[0113] The technical solution of this invention identifies candidate target objects in a first image matched in a first region, scores the candidate target objects for whether they carry a target object, and determines the target object to be detected from the candidate target objects based on the behavior score. If the target object to be detected is detected in a second image matched in a second region, it indicates that a target object exists in the second region. This solves the problems of low recognition rate, high false alarm rate, and limited detection range in existing technologies that directly identify target objects in the second region using physical sensors or image recognition. It achieves accurate identification of target objects entering a specific region, reducing security risks.

[0114] Based on the above embodiments, optionally, the candidate target object determination module 310 includes:

[0115] The perimeter detection unit is used to perform perimeter detection on the first region using an imaging device matched with the first region;

[0116] The candidate target object determination unit is used to determine the target object as a candidate target object if it is determined from the first image captured by the shooting device that the target object has entered the first area and the dwell time in the first area is greater than or equal to a preset time threshold.

[0117] Based on the above embodiments, optionally, the target object determination module 320 includes:

[0118] The behavior scoring determination unit is used to determine the behavior score of the candidate target object based on at least one of the following: the carrying object score, the dwell time score, and the action posture score.

[0119] The carrying object score is determined based on the object characteristics of the object carried by the candidate target object, the dwell time score is determined based on the dwell time of the candidate target object in the first area, and the action posture score is determined based on the action posture of the candidate target object in the first area.

[0120] Based on the above embodiments, optionally, the target object determination module 320 includes:

[0121] The target object determination unit is used to determine the candidate target object as the target object to be detected if the behavior score of the candidate target object is determined to be greater than or equal to a preset score threshold.

[0122] Optionally, based on the above embodiments, the apparatus further includes:

[0123] The image determination module is used to determine a first image that matches a first region, a second image that matches a second region, and a third image that matches a path between the first region and the second region;

[0124] The motion trajectory determination module is used to determine the motion trajectory of the target object to be detected based on at least one of the first image, the second image, and the third image.

[0125] Optionally, based on the above embodiments, the apparatus further includes:

[0126] The target object prompting module is used to provide a pre-announcement prompt that the target object is entering the second area if the movement trend of the target object to be detected is determined to be entering the second area based on the movement trajectory.

[0127] Optionally, based on the above embodiments, the apparatus further includes:

[0128] The target object deletion module is used to delete the target object if the movement trend of the target object is determined to be other than entering the second region based on the movement trajectory, and the duration of the movement trajectory is greater than or equal to a preset duration threshold.

[0129] The target object detection device provided in the embodiments of the present invention can execute the target object detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0130] Example 4

[0131] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0132] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0133] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0134] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as methods for detecting target objects.

[0135] In some embodiments, the target object detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the target object detection method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the target object detection method by any other suitable means (e.g., by means of firmware).

[0136] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0137] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0138] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0139] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0140] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0141] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0142] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0143] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting a target object, characterized in that, include: Based on the first image that matches the first region, at least one candidate target object is determined; Determine the behavior score of at least one candidate target object, and determine the target object to be detected among the candidate target objects based on the behavior score of the candidate target objects; The behavior score is used to indicate the probability that the target object carries the target object. If the target object is detected in a second image that matches the second region, then the target object is determined to have entered the second region.

2. The method according to claim 1, characterized in that, Based on a first image that matches a first region, at least one candidate target object is identified, including: Perimeter detection of the first region is performed using a shooting device matched with the first region; If, based on the first image captured by the imaging device, it is determined that the target object has entered the first area and its stay time in the first area is greater than or equal to a preset time threshold, then the target object is considered a candidate target object.

3. The method according to claim 1, characterized in that, Determine the behavioral score of at least one candidate target object, including: The behavioral score of the candidate target object is determined based on at least one of the following: the score of the object carried, the score of the dwell time, and the score of the action posture. The carrying object score is determined based on the object characteristics of the object carried by the candidate target object, the dwell time score is determined based on the dwell time of the candidate target object in the first area, and the action posture score is determined based on the action posture of the candidate target object in the first area.

4. The method according to claim 1, characterized in that, Based on the behavioral scores of the candidate target objects, the target objects to be detected are determined from the candidate target objects, including: If the behavior score of a candidate target object is determined to be greater than or equal to a preset score threshold, then the candidate target object is taken as the target object to be detected.

5. The method according to claim 1, characterized in that, After identifying the target object to be detected from the candidate target objects, the process also includes: Identify a first image that matches a first region, a second image that matches a second region, and a third image that matches the path between the first and second regions; The motion trajectory of the target object to be detected is determined based on at least one of the first image, the second image, and the third image.

6. The method according to claim 5, characterized in that, After determining the motion trajectory of the target object to be detected, the process also includes: If the movement trend of the target object to be detected is determined to be entering the second region based on the movement trajectory, then a warning prompt is issued that the target object is entering the second region.

7. The method according to claim 5, characterized in that, After determining the motion trajectory of the target object to be detected, the process also includes: If the movement trend of the target object to be detected is determined to be other than entering the second region based on the movement trajectory, and the duration of the movement trajectory is greater than or equal to a preset duration threshold, then the target object to be detected is deleted.

8. A device for detecting a target object, characterized in that, include: The candidate target object determination module is used to determine at least one candidate target object based on a first image that matches the first region; The target object determination module is used to determine the behavior score of at least one candidate target object, and determine the target object to be detected from the candidate target objects based on the behavior score of the candidate target objects; The behavior score is used to indicate the probability that the target object carries the target object. The target object determination module is used to determine that the target object has entered the second region if the target object to be detected is detected in the second image that matches the second region.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the target object detection method as described in any one of claims 1-7.

10. A storage medium for storing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the target object detection method as described in any one of claims 1-7.

Images