Method, device and storage medium for detecting leftover objects

Abstract

The present disclosure provides a method, an apparatus, and a storage medium for detecting a left-behind object, which relates to the field of video surveillance and can solve the problem of low accuracy in detecting left-behind objects in related technologies. The method includes: obtaining a to-be-detected image of a target area at a first moment; determining whether there is a foreground image in the to-be-detected image according to a foreground image determination model; when there is a foreground image in the to-be-detected image and the foreground image meets a first preset condition, inputting the foreground image and at least one comparison image into a preset tracking model to obtain at least one tracking result; the at least one comparison image is an image of the target area within a second time period, and the second time period is a time period after the first moment; detecting whether the foreground object is an object left behind in the target area according to the at least one tracking result. The present disclosure can improve the accuracy of detecting left-behind objects.

Description

Technical Field

[0001] The present disclosure relates to the field of video surveillance, and particularly to a method, apparatus, and storage medium for detecting left-behind objects. Background Art

[0002] In the field of video surveillance, the detection of left-behind objects can be applied to various scenarios. For example, detecting left-behind objects in public areas can promptly discover when a person has lost an item and trigger an alarm to avoid property losses. Detecting object retention in key areas can promptly discover object blockage problems and trigger an alarm to eliminate potential safety hazards.

[0003] Currently, related technologies typically determine whether a foreground object appears by detecting whether there are significant changes in the pixel values of the pixel points in the image. When there are sudden changes in the pixel values in the surveillance image due to factors such as changes in lighting, this solution will identify the pixel points with sudden changes in pixel values as foreground pixel points, resulting in low accuracy in the detection of foreground objects. Summary of the Invention

[0004] The present disclosure provides a method, apparatus, and storage medium for detecting left-behind objects, which can improve the accuracy of left-behind object detection.

[0005] To achieve the above object, the present disclosure adopts the following technical solutions:

[0006] On the one hand, a method for detecting left-behind objects is provided. The method includes: obtaining a first image to be detected of a target area at a first moment; determining whether there is a foreground image in the first image to be detected according to a foreground image determination model; the foreground image is an image corresponding to a foreground object within the target area; when there is a foreground image in the first image to be detected and the foreground image meets a first preset condition, inputting the foreground image and at least one comparison image into a preset tracking model to obtain at least one tracking result; the at least one comparison image is an image of the target area within a second time period, the second time period is a time period after the first moment, and the first preset condition includes at least one of a ratio of an area of the foreground image to an area of the first image to be detected being greater than a first threshold and a number of pixel points of the foreground image being greater than a second threshold; detecting whether the foreground object is an object left behind in the target area according to the at least one tracking result.

[0007] Based on the above technical solution, the detection device for the remaining object in the present disclosure determines whether there is a foreground image in the image by obtaining the detection image of the target area at the first moment and determining the model based on the foreground image. When there is a foreground image in the detection image and the foreground image meets the first preset condition, it indicates that the foreground object corresponding to the foreground image is the potential remaining object to be detected in the present disclosure. Therefore, the present disclosure further needs to make a further judgment through a preset tracking model. The detection device for the remaining object inputs the foreground image and at least one comparison image into the preset tracking model to obtain at least one tracking result, and detects whether the foreground object is an object remaining in the target area according to the at least one tracking result. Since the above at least one comparison image is the image of the target area within the second time period after the first moment, the present disclosure can determine whether the foreground object corresponding to the foreground image stays in the target area for more than a certain duration, so as to determine whether the foreground object is an object remaining in the target area. Compared with the related art solution of detecting the remaining object only by detecting whether the pixel points in the picture change, the present disclosure can avoid the problem that the detection of the foreground object is deviated due to the sudden change of the pixel points in the area caused by factors such as light change, and improves the accuracy of the remaining object detection.

[0008] In some embodiments, the foreground image determination model includes one or more sub-models corresponding to the positions of each pixel point in the background image of the target area. The method includes: detecting whether one or more sub-models of the second pixel points corresponding to each first pixel point of the first image to be detected match, where the second pixel points are the pixel points corresponding to the first pixel points in the background image; when none of the one or more sub-models of at least one first pixel point and the corresponding second pixel point match, determining that there is a foreground image in the first image to be detected; when the sub-models of all first pixel points and the corresponding second pixel points match, determining that there is no foreground image in the first image to be detected.

[0009] In some embodiments, the method includes: determining the parameter value of any first pixel point of the first image to be detected and the parameter intervals of each sub-model in the one or more sub-models of the second pixel point corresponding to the first pixel point; when the parameter value of the first pixel point is within the parameter interval of the first sub-model, determining that the first pixel point matches the first sub-model; the first sub-model is at least one sub-model in the one or more sub-models of the second pixel point corresponding to the first pixel point; when the parameter value of the first pixel point is outside the parameter interval of the first sub-model, determining that the first pixel point does not match the first sub-model.

[0010] In some embodiments, the preset tracking model includes at least one neural network model; the method includes: inputting the foreground image into the first neural network model to obtain the first image feature of the foreground image, where the first neural network model is any one of the at least one neural network model; inputting each of the at least one comparison image into the second neural network model to obtain the second image feature corresponding to each comparison image in the at least one comparison image, where the second neural network model is any one of the at least one neural network model other than the first neural network; comparing the first image feature with the second image feature corresponding to each comparison image to obtain at least one tracking result.

[0011] In some embodiments, at least one tracking result includes at least one of a first parameter value, the range of the tracking image, and the tracking position; wherein, the tracking image is the sub-image in the comparison image with the highest similarity to the foreground image, the first parameter value is used to represent the similarity between the foreground image and the tracking image, the range of the tracking image is the area occupied by the tracking image in the comparison image, and the tracking position is the position of the tracking image in the comparison image.

[0012] In some embodiments, the method includes: when at least one tracking result meets the second preset condition, determining that the foreground object is an object left in the target area, where the second preset condition includes at least one limiting condition, and the at least one limiting condition corresponds to the at least one tracking result; when any one of the at least one tracking result does not meet the second preset condition, determining that the foreground object is not an object left in the target area.

[0013] In some embodiments, the method includes: when the first parameter value is greater than the third threshold, and / or the range of the tracking image is greater than the fourth threshold, and / or the tracking position is within at least one preset range in the comparison image, determining that the foreground object is an object left in the target area.

[0014] In some embodiments, before inputting the foreground image and the at least one comparison image into the preset tracking model to obtain at least one tracking result, the method further includes: obtaining one or more background images and determining the sub-background images of each background image in the one or more background images, where the position of the sub-background image in the background image corresponds to the position of the foreground image in the first image to be detected; inputting the foreground image and the one or more sub-background images into the verification model to obtain at least one second parameter value; the second parameter value is used to represent the similarity between the foreground image and the sub-background image; when at least one second parameter value is less than the fifth threshold, determining that there is a foreground image in the first image to be detected.

[0015] In some embodiments, the method further includes: updating the foreground image determination model when there is no foreground image in the first image to be detected, or the foreground image does not meet the first preset condition, or the foreground object is not an object left in the target area.

[0016] In some embodiments, the method includes: determining an updated image; the updated image being the first image to be detected; and updating the foreground image determination model according to the updated image to obtain an updated foreground image determination model.

[0017] In some embodiments, the method includes: for each third pixel point in the updated image, detecting whether there is a sub-model in one or more sub-models corresponding to the second pixel point that matches the third pixel point, where the second pixel point is the pixel point in the background image corresponding to the position of the third pixel point; when there is a second sub-model in the one or more sub-models, increasing the weight value of the second sub-model and decreasing the weight value of the third sub-model, where the second sub-model is the sub-model in the one or more sub-models that matches the third pixel point, and the third sub-model is other sub-models in the one or more sub-models except the second sub-model; and obtaining an updated foreground image determination model according to the increased weight value of the second sub-model and the decreased weight value of the third sub-model.

[0018] In some embodiments, the method includes: when there is no second sub-model in the one or more sub-models, generating a fourth sub-model according to the third pixel point and replacing the sub-model with the smallest weight value in the one or more sub-models with the fourth sub-model to obtain an updated foreground image determination model.

[0019] In some embodiments, the method further includes: obtaining a second image to be detected of the target area at a second moment; the second moment being a moment after the first moment; determining whether there is a foreground image in the second image to be detected according to the foreground image determination model; when there is a foreground image in the second image to be detected and the foreground image meets the first preset condition, inputting the foreground image and at least one second comparison image into a preset tracking model to obtain at least one tracking result; the at least one second comparison image being an image of the target area within a third time period, the third time period being a time period after the second moment, and the first preset condition including at least one of a ratio of an area of the foreground image to an area of the second image to be detected being greater than a first threshold and a number of pixel points of the foreground image being greater than a second threshold; and detecting whether the foreground object is an object left in the target area according to the at least one tracking result.

[0020] In some embodiments, the method further includes: outputting a prompt message when the foreground object is an object left in the target area.

[0021] On the other hand, a detection device for a legacy object is provided, including: a processing unit configured to obtain a to-be-detected image of a target area at a first moment; the processing unit is further configured to determine whether there is a foreground image in the to-be-detected image according to a foreground image determination model; the foreground image is an image corresponding to a foreground object within the target area; in the case where there is a foreground image in the to-be-detected image and the foreground image satisfies a first preset condition, the processing unit is further configured to input the foreground image and at least one comparison image into a preset tracking model to obtain at least one tracking result; the at least one comparison image is an image of the target area within a second time period, the second time period is a time period after the first moment, and the first preset condition includes at least one of a ratio of an area of the foreground image to an area of the to-be-detected image being greater than a first threshold and a number of pixel points of the foreground image being greater than a second threshold; the processing unit is further configured to detect whether the foreground object is an object left in the target area according to the at least one tracking result.

[0022] In some embodiments, the foreground image determination model includes one or more sub-models corresponding to the position of each pixel point in the background image of the target area, the background image does not include the foreground image, and the processing unit is configured to: detect whether one or more sub-models of a second pixel point corresponding to each first pixel point of the first to-be-detected image match, the second pixel point being the pixel point in the background image corresponding to the position of the first pixel point; in the case where one or more sub-models of at least one first pixel point and the corresponding second pixel point do not match, determine that there is a foreground image in the first to-be-detected image; in the case where sub-models of all first pixel points and the corresponding second pixel points match, determine that there is no foreground image in the first to-be-detected image.

[0023] In some embodiments, the processing unit is configured to: determine a parameter value of any first pixel point of the first to-be-detected image and a parameter interval of each sub-model among one or more sub-models of the second pixel point corresponding to the first pixel point; in the case where the parameter value of the first pixel point is within the parameter interval of the first sub-model, determine that the first pixel point matches the first sub-model; the first sub-model is at least one sub-model among one or more sub-models of the second pixel point corresponding to the first pixel point; in the case where the parameter value of the first pixel point is outside the parameter interval of the first sub-model, determine that the first pixel point does not match the first sub-model.

[0024] In some embodiments, the preset tracking model includes at least one neural network model; the processing unit is configured to: input the foreground image into the first neural network model to obtain the first image feature of the foreground image, where the first neural network model is any one of the at least one neural network model; input each of the at least one comparison image into the second neural network model to obtain the second image feature corresponding to each comparison image in the at least one comparison image, where the second neural network model is any one of the at least one neural network model other than the first neural network; compare the first image feature with the second image feature corresponding to each comparison image to obtain at least one tracking result.

[0025] In some embodiments, at least one tracking result includes at least one of a first parameter value, the range of the tracking image, and the tracking position; wherein, the tracking image is the sub-image in the comparison image with the highest similarity to the foreground image, the first parameter value is used to represent the similarity between the foreground image and the tracking image, the range of the tracking image is the area occupied by the tracking image in the comparison image, and the tracking position is the position of the tracking image in the comparison image.

[0026] In some embodiments, the processing unit is configured to: determine that the foreground object is an object left in the target area when at least one tracking result meets the second preset condition, where the second preset condition includes at least one limiting condition, and the at least one limiting condition corresponds to the at least one tracking result; determine that the foreground object is not an object left in the target area when any one of the at least one tracking result does not meet the second preset condition.

[0027] In some embodiments, the processing unit is configured to: determine that the foreground object is an object left in the target area when the first parameter value is greater than the third threshold, and / or the range of the tracking image is greater than the fourth threshold, and / or the tracking position is within at least one preset range in the comparison image.

[0028] In some embodiments, the processing unit is further configured to: obtain one or more background images and determine the sub-background images of each of the one or more background images, where the position of the sub-background image in the background image corresponds to the position of the foreground image in the first image to be detected; input the foreground image and the one or more sub-background images into the verification model to obtain at least one second parameter value; the second parameter value is used to represent the similarity between the foreground image and the sub-background image; determine that there is a foreground image in the first image to be detected when at least one second parameter value is less than the fifth threshold.

[0029] In some embodiments, when there is no foreground image in the first image to be detected, or the foreground image does not meet the first preset condition, or the foreground object is not an object left in the target area, the processing unit is further configured to update the foreground image determination model.

[0030] In some embodiments, the processing unit is configured to: determine an updated image; the updated image is the first image to be detected; update the foreground image determination model according to the updated image to obtain an updated foreground image determination model.

[0031] In some embodiments, the foreground image determination model includes one or more sub-models corresponding to the positions of each pixel point in the background image of the target area, and each sub-model corresponds to a weight value; the processing unit is configured to: for each third pixel point in the updated image, detect whether there is a sub-model in one or more sub-models corresponding to the second pixel point that matches the third pixel point, where the second pixel point is the pixel point in the background image corresponding to the position of the third pixel point; when there is a second sub-model in one or more sub-models, increase the weight value of the second sub-model and decrease the weight value of the third sub-model, where the second sub-model is the sub-model in one or more sub-models that matches the third pixel point, and the third sub-model is other sub-models in one or more sub-models except the second sub-model; obtain an updated foreground image determination model according to the increased weight value of the second sub-model and the decreased weight value of the third sub-model.

[0032] In some embodiments, the processing unit is configured to: when there is no second sub-model in one or more sub-models, generate a fourth sub-model according to the third pixel point, and replace the sub-model with the smallest weight value in one or more sub-models with the fourth sub-model to obtain an updated foreground image determination model.

[0033] In some embodiments, the processing unit is configured to: obtain a second image to be detected of the target area at a second moment; the second moment is a moment after the first moment; determine whether there is a foreground image in the second image to be detected according to the foreground image determination model; when there is a foreground image in the second image to be detected and the foreground image meets the first preset condition, input the foreground image and at least one second comparison image into a preset tracking model to obtain at least one tracking result; the at least one second comparison image is an image of the target area within a third time period, and the third time period is a time period after the second moment, and the first preset condition includes at least one of a ratio of the area of the foreground image to the area of the second image to be detected being greater than a first threshold and the number of pixel points of the foreground image being greater than a second threshold; detect whether the foreground object is an object left in the target area according to the at least one tracking result.

[0034] In some embodiments, the detection device for the legacy object further includes a communication unit, which is configured to output a prompt message when the foreground object is an object left in the target area.

[0035] In another aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores computer program instructions, which, when running on a computer (e.g., the detection device for the legacy object), cause the computer to execute the method for detecting the legacy object as described in any of the above embodiments.

[0036] In yet another aspect, a computer program product is provided. The computer program product includes computer program instructions, which, when executed on a computer (e.g., the detection device for the legacy object), cause the computer to execute the method for detecting the legacy object as described in any of the above embodiments.

[0037] In yet another aspect, a computer program is provided. When the computer program is executed on a computer (e.g., the detection device for the legacy object), the computer program causes the computer to execute the method for detecting the legacy object as described in any of the above embodiments.

[0038] In yet another aspect, a chip is provided. The chip includes a processor and a communication interface, and the communication interface is coupled to the processor. The processor is configured to run a computer program or instructions to implement the method for detecting the legacy object as described in any of the above embodiments.

[0039] Specifically, the chip provided in the present disclosure further includes a memory for storing computer programs or instructions.

[0040] It should be noted that the above computer instructions may be stored in whole or in part on a computer-readable storage medium. The computer-readable storage medium may be packaged together with the processor of the device or separately packaged from the processor of the device, and the present disclosure does not limit this.

[0041] In yet another aspect, a detection system for the legacy object is provided, including: a detection device for the legacy object and at least one camera device, where the detection device for the legacy object is configured to execute the method for detecting the legacy object as described in any of the above embodiments.

[0042] In the present disclosure, the name of the above detection device for the legacy object does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the functions of each device or functional module are similar to those of the present disclosure and fall within the scope of the claims of the present disclosure and their equivalent technologies. Description of the Drawings

[0043] To more clearly illustrate the technical solutions in the present disclosure, the following will briefly introduce the drawings required for use in some embodiments of the present disclosure. Obviously, the drawings in the following description are only the drawings of some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings. In addition, the drawings in the following description can be regarded as schematic diagrams and are not limitations on the actual dimensions of the products, the actual processes of the methods, the actual timings of the signals, etc. involved in the embodiments of the present disclosure.

[0044] Figure 1 It is a structural diagram of a detection system for a left-behind object provided according to some embodiments;

[0045] Figure 2 It is a flowchart of a detection method for a left-behind object provided according to some embodiments;

[0046] Figure 3 It is a scene diagram of a background image and a first image to be detected provided according to some embodiments;

[0047] Figure 4 It is a scene diagram of a first image to be detected and a comparison image provided according to some embodiments;

[0048] Figure 5 It is a flowchart of another detection method for a left-behind object provided according to some embodiments;

[0049] Figure 6 It is a flowchart of another detection method for a left-behind object provided according to some embodiments;

[0050] Figure 7 It is a structural diagram of a preset tracking model provided according to some embodiments;

[0051] Figure 8 It is a flowchart of another detection method for a left-behind object provided according to some embodiments;

[0052] Figure 9 It is a flowchart of another detection method for a left-behind object provided according to some embodiments;

[0053] Figure 10 It is a scene diagram of another background image and a first image to be detected provided according to some embodiments;

[0054] Figure 11 It is a flowchart of another detection method for a left-behind object provided according to some embodiments;

[0055] Figure 12 It is a flowchart of another detection method for a left-behind object provided according to some embodiments;

[0056] Figure 13 Structural diagram of a detection device for a left-behind object provided according to some embodiments;

[0057] Figure 14 Structural diagram of another detection device for a left-behind object provided according to some embodiments. Detailed implementation manners

[0058] Next, in conjunction with the accompanying drawings, the technical solutions in some embodiments of the present disclosure will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments provided by the present disclosure, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present disclosure.

[0059] Unless otherwise required by the context, throughout the specification and claims, the term "comprise" and its other forms, such as the third-person singular form "comprises" and the present participle form "comprising", are interpreted as open and inclusive meanings, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "example", "specific example", or "some examples", etc. are intended to indicate that the specific features, structures, materials, or characteristics related to the embodiment or example are included in at least one embodiment or example of the present disclosure. The schematic representations of the above terms are not necessarily referring to the same embodiment or example. In addition, the specific features, structures, materials, or characteristics may be included in any one or more embodiments or examples in any appropriate manner.

[0060] Hereinafter, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present disclosure, unless otherwise stated, the meaning of "a plurality" is two or more.

[0061] In describing some embodiments, the expressions "coupled" and "connected" and their derivatives may be used. For example, in describing some embodiments, the term "connected" may be used to indicate that two or more components have direct physical or electrical contact with each other. Another example is that in describing some embodiments, the term "coupled" may be used to indicate that two or more components have direct physical or electrical contact. However, the term "coupled" or "communicatively coupled" may also mean that two or more components do not have direct contact with each other, but still cooperate or interact with each other. The embodiments disclosed herein are not necessarily limited to the content herein.

[0062] "At least one of A, B, and C" has the same meaning as "at least one of A, B, or C", and both include the following combinations of A, B, and C: only A, only B, only C, the combination of A and B, the combination of A and C, the combination of B and C, and the combination of A, B, and C.

[0063] "A and / or B" includes the following three combinations: only A, only B, and the combination of A and B.

[0064] As used herein, depending on the context, the term "if" is optionally interpreted to mean "when", "while", "in response to determining", or "in response to detecting". Similarly, depending on the context, the phrase "if it is determined that..." or "if [the stated condition or event] is detected" is optionally interpreted to mean "when it is determined that...", "in response to determining...", "when [the stated condition or event] is detected", or "in response to detecting [the stated condition or event]".

[0065] The use of "suitable for" or "configured to" herein means open and inclusive language, which does not exclude devices that are suitable for or configured to perform additional tasks or steps.

[0066] In addition, the use of "based on" means open and inclusive, because a process, step, calculation, or other action "based on" one or more of the stated conditions or values may, in practice, be based on additional conditions or values beyond the stated ones.

[0067] As used herein, "about", "substantially", or "approximately" includes the stated value and the average value within an acceptable deviation range of the specific value, where the acceptable deviation range is determined by a person of ordinary skill in the art considering the measurement being discussed and the errors associated with the measurement of the specific quantity (i.e., the limitations of the measurement system).

[0068] Hereinafter, the nouns related to the embodiments of the present disclosure are explained to facilitate the understanding of the readers.

[0069] (1) Gaussian distribution

[0070] The Gaussian distribution, also known as the normal distribution, is used to represent the probability distribution of a variable. Its corresponding function graph has the characteristics of being low on both sides, high in the middle, and symmetric on the left and right. The Gaussian distribution has two parameters, the mean and the variance. The mean is the value of the variable when the probability density of the Gaussian distribution is the largest, and the variance is used to represent the decline amplitude of the function graph. The variance is the square of the standard deviation, so the variance and the standard deviation can be converted to each other.

[0071] (2) Neural networks

[0072] Neural networks (NNs), also known as artificial neural networks (ANNs), are mathematical model algorithms that imitate the behavioral characteristics of animal neural networks and perform distributed parallel information processing. Neural networks include deep learning networks, such as convolutional neural networks (CNN), long short-term memory (LSTM), etc.

[0073] (3) Region of interest (ROI)

[0074] The region of interest is the area to be processed that is outlined by a box, a circle, or an irregular shape from the image to be processed. For example, the foreground image determined from the first image to be detected in this disclosure is the ROI required by this disclosure.

[0075] Next, the implementation manners of the embodiments of the present disclosure will be described in detail with reference to the accompanying drawings of the specification.

[0076] As Figure 1 shown, Figure 1 is a schematic structural diagram of a left-behind object detection system 10 provided according to some embodiments. The left-behind object detection system 10 includes: a detection device 101 for left-behind objects and at least one camera device 102 ( Figure 1 only one camera device is shown). The detection device 101 for left-behind objects is connected to at least one camera device 102 through a communication link. The communication link can be a wired communication link or a wireless communication link, which is not limited here.

[0077] Among them, the camera device 102 is used to acquire image data of the target area and send the image data to the detection device 101 for left-behind objects. Correspondingly, the detection device 101 for left-behind objects receives the image data sent by the camera device 102.

[0078] Exemplarily, the target area may be an area that needs to be monitored, such as an airport waiting area, a fire passage, a sewer, a building staircase passage, etc.

[0079] In a possible implementation, the imaging device 102 may acquire image data of the target area in real time and send it to the detection device 101 for left-behind objects, or the imaging device 102 may also acquire image data of the target area at a preset frequency and send it to the detection device 101 for left-behind objects.

[0080] The imaging device 102 in the embodiments of the present disclosure is a device that represents image information as an analog signal or a digital signal through a photosensor, and can be deployed on land, including indoor or outdoor, handheld or vehicle-mounted. It can also be deployed on the water surface (such as a ship, etc.). It can also be deployed in the air (such as an airplane, a balloon, a satellite, etc.). For example, the imaging device 102 includes a camera, a video camera, a camera. The imaging device 102 may also be a device with imaging functions. For example, the imaging device 102 may be a mobile phone, a tablet computer, a notebook computer, a handheld computer, a wearable device (such as a smart watch, a smart bracelet, a pedometer, etc.), a vehicle-mounted device, a flight device (such as a smart robot, a hot air balloon, a drone, an airplane), etc. with imaging functions.

[0081] Exemplarily, the imaging device 102 in the embodiments of the present disclosure may also be an infrared imager or a night vision device for acquiring image information of a dark area.

[0082] Among them, the detection device 101 for left-behind objects is used to receive image data from the imaging device 102 and detect whether there are left-behind items in the target area according to the image data.

[0083] The detection device 101 for left-behind objects in the embodiments of the present disclosure may be a server, including:

[0084] A processor, which may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present disclosure solution.

[0085] A transceiver, which may be any device of the transceiver type for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0086] The memory can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory can exist independently and be connected to the processor through a communication line. The memory can also be integrated with the processor.

[0087] The detection device 101 for the legacy object in the embodiments of the present disclosure can also be a part of the device coupled to the server, such as a chip system in the server.

[0088] It should be noted that the embodiments of the present disclosure can learn from or refer to each other. For example, for the same or similar steps, among the method embodiments, system embodiments, and device embodiments, they can all refer to each other without limitation.

[0089] In the field of video surveillance, the detection of legacy objects can be applied to various scenarios. For example, if a legacy object is detected in a public area, it can be timely discovered that a person has lost an item and an alarm can be triggered to avoid causing property losses to the user. Another example is that if an object is detected staying in a key area (such as a fire escape), the problem of object blockage can be timely discovered and an alarm can be triggered to eliminate potential safety hazards.

[0090] Generally, the Gaussian mixture model (GMM) is usually adopted to detect the foreground objects appearing in the surveillance video. However, this solution detects whether a foreground object appears by detecting whether the pixel values of the pixel points in the video change significantly. When the pixel values in the surveillance video mutate due to factors such as light changes, this solution will confirm the pixel points with mutated pixel values as foreground pixel points, resulting in low detection accuracy of the foreground objects.

[0091] In view of this, the present disclosure provides a method for detecting legacy objects.

[0092] As Figure 2 shown, Figure 2A method for detecting a left-behind object provided according to some embodiments, the method comprising the following steps:

[0093] S201. The detection device for the left-behind object acquires a first image to be detected of a target area at a first moment.

[0094] Among them, the detection device for the left-behind object may be Figure 1 the detection device 101 in, or may be a component of the detection device 101, such as a chip, etc. The target area is the area detected by the detection device for the left-behind object, and the first image to be detected is an image corresponding to the target area.

[0095] In a possible implementation manner, the detection device may acquire the first image to be detected of the target area at the first moment through an imaging device. The imaging device may be Figure 1 the imaging device 102 in.

[0096] In an example, the detection device for the left-behind object may capture an image of the target area through the imaging device and obtain the first image to be detected of the target area therefrom.

[0097] In another example, the detection device for the left-behind object may periodically capture images of the target area through the imaging device and obtain the first image to be detected of the target area therefrom.

[0098] In a possible implementation manner, when there are changed pixel points in the image of the target area acquired by the detection device for the left-behind object at the first moment (such as the current moment) compared with the image before the first moment, the detection device for the left-behind object uses this image as the first image to be detected of the target area.

[0099] S202. The detection device for the left-behind object determines whether there is a foreground image in the first image to be detected according to the foreground image determination model.

[0100] Among them, the foreground image determination model is used to determine whether there is a foreground image in the image. The foreground image is an image corresponding to a foreground object in the target area.

[0101] It should be noted that the foreground image determination model may be pre-configured in the detection device for the left-behind object, or may be obtained by the detection device for the left-behind object from other devices / servers. For example, the foreground image determination model may be trained by multiple background images. The specific training process may refer to the following description.

[0102] It should be pointed out that the foreground object in the present disclosure refers to an objectively existing physical substance in nature, such as a person, an animal, a plant, a vehicle, a commodity, etc.

[0103] Exemplarily, such as Figure 3As shown Figure 3 where a in Figure 3 is the background image of the target area Figure 3 and b in Figure 3 is the first image to be detected in the target area. Among them Figure 3 there is a foreground object 30 in b in Figure 3 . The detection device for the remaining object determines the foreground image corresponding to the foreground object 30 in the first image to be detected according to the foreground image determination model (the image framed by the dotted line in the figure).

[0104] In a possible implementation, the foreground image determination model can determine whether there is a foreground image according to the change situation of the pixel points in the first image to be detected and the pixel points at the corresponding positions in the background image.

[0105] Among them, the pixel points can be represented by pixel values or by colors.

[0106] Exemplarily, taking the pixel points represented by pixel values as an example, the detection device for the remaining object determines the change situation of the pixel values of each pixel point in the first image to be detected and the pixel values of the pixel points at the corresponding positions in the background image according to the foreground image determination model. If there are pixel points with changed pixel values in the first image to be detected, it is determined that there is a foreground image in the first image to be detected. On the contrary, if there are no pixel points with changed pixel values in the first image to be detected, it is determined that there is no foreground image in the first image to be detected. The pixel points with changed pixel values are the foreground pixel points.

[0107] In a possible implementation, the detection device for the remaining object determines the range of the foreground image by calculating the connected regions of the pixel points and morphological processing. The range of the foreground image is the area occupied by the foreground image in the first image to be detected.

[0108] Exemplarily, the methods for calculating the connected regions include the four-connected calculation method and the eight-connected calculation method. The four-connected calculation method means connecting the foreground pixel points in the upper, lower, left, and right four directions of the position where any foreground pixel point is located to obtain the foreground image.

[0109] The morphological processing includes methods such as noise elimination and erosion operation. The specific process can refer to the related technology, and the present disclosure does not limit this.

[0110] In the case where there is no foreground image in the first image to be detected, the detection device for the remaining object can update the foreground image determination model.

[0111] The process of updating the foreground image determination model can refer to the following description and will not be elaborated here.

[0112] S203. When there is a foreground image in the first image to be detected and the foreground image meets the first preset condition, the detection device for the left-behind object inputs the foreground image and at least one comparison image into a preset tracking model to obtain at least one tracking result.

[0113] Wherein, the at least one comparison image is an image of the target area in the second time period, and the second time period is the time period after the first moment. The at least one comparison image may be a continuous image of the target area in the second time period or a discontinuous image in the second time period.

[0114] Wherein, the first preset condition may include one or more limiting conditions, and the limiting conditions are used to determine whether to perform tracking detection according to the foreground image. For example, the first preset condition may include at least one of a ratio of the area of the foreground image to the area of the first image to be detected being greater than a first threshold and the number of pixel points of the foreground image being greater than a second threshold. The first threshold is a ratio threshold, and the second threshold is a pixel point number threshold. The first threshold and the second threshold can be set according to actual situations, and the present disclosure does not limit this.

[0115] It should be noted that when there is a foreground image in the first image to be detected, at this time, the foreground object corresponding to the foreground image may not be the left-behind object to be detected in the present disclosure. For example, the foreground object may be a tiny object such as a falling leaf. Therefore, the detection device for the left-behind object can determine whether the foreground image meets the first preset condition, and if it meets the first preset condition, then further track the foreground object in the target area in the time period after the first moment. In this way, the detection device for the left-behind object can exclude tiny objects in the target area and avoid detection errors.

[0116] In addition, when there are multiple foreground images in the first image to be detected and the multiple foreground images meet the first preset condition, the detection device for the left-behind object can respectively perform object tracking on the objects corresponding to the multiple foreground images.

[0117] Wherein, the preset tracking model is used to determine, according to the foreground image, the sub-image with the highest similarity to the foreground image in the comparison image as the tracking image, so as to obtain the corresponding tracking result.

[0118] Exemplarily, as Figure 4 shown, Figure 4 a in is the first image to be detected at the first moment, in which there is a foreground image 40. Figure 4 b in and Figure 4 c in are comparison images in the second time period, in which there is a tracking image 41. The detection device for the left-behind object inputs the foreground image 40 and Figure 4Input b and c into a preset tracking model to determine Figure 4 the sub-image in b and c with the highest similarity to the foreground image 40 in Figure 4 as the tracking image 41, thereby obtaining the corresponding tracking result. The tracking result is used to represent the remaining situation of the foreground object corresponding to the foreground image 40 in the comparison image.

[0119] It should be noted that each comparison image in at least one comparison image in the present disclosure corresponds to one or more tracking results. Therefore, the at least one tracking result obtained above includes one or more tracking results corresponding to each comparison image. Each comparison image includes a plurality of sub-images.

[0120] Among them, at least one tracking result includes at least one of a first parameter value, the range of the tracking image, and the tracking position. The tracking image can be the sub-image in the comparison image with the highest similarity to the foreground image. The first parameter value is used to represent the similarity between the foreground image and the tracking image. The range of the tracking image is the area occupied by the tracking image in the comparison image, and the tracking position is the position of the tracking image in the comparison image.

[0121] The area occupied by the tracking image in the comparison image can be represented by the proportion of the area occupied by the tracking image in the comparison image, or by the number of pixel points in the area occupied by the tracking image in the comparison image.

[0122] In the case where the foreground image does not meet the first preset condition, the remaining object detection device can update the foreground image determination model.

[0123] The process of updating the foreground image determination model can refer to the following description and will not be elaborated here.

[0124] S204. The remaining object detection device detects whether the foreground object is an object remaining in the target area according to at least one tracking result.

[0125] In a possible implementation, the remaining object detection device can determine whether the at least one tracking result meets the second preset condition. When the at least one tracking result meets the second preset condition, the remaining object detection device determines that the foreground object is an object remaining in the target area. Conversely, when any one of the at least one tracking result does not meet the second preset condition, the remaining object detection device determines that the foreground object is not an object remaining in the target area.

[0126] Among them, the second preset condition can include at least one limiting condition, and the at least one limiting condition corresponds to the at least one tracking result. The at least one limiting condition includes at least one of the first parameter value being greater than a third threshold, the range of the tracking image being greater than a fourth threshold, and the tracking position being within at least one preset range in the comparison image.

[0127] The specific judgment process can refer to the following description and will not be elaborated here.

[0128] Based on the above technical solution, the detection device for left objects in the present disclosure determines whether there is a foreground image in the image by obtaining the detection image of the target area at the first moment and determining the model according to the foreground image. When there is a foreground image in the detection image and the foreground image meets the first preset condition, it indicates that the foreground object corresponding to the foreground image is a potential left object to be detected in the present disclosure. Therefore, the present disclosure also needs to make a further judgment through a preset tracking model. The detection device for left objects inputs the foreground image and at least one comparison image into the preset tracking model to obtain at least one tracking result, and detects whether the foreground object is an object left in the target area according to the at least one tracking result. Since the at least one comparison image is an image of the target area within the second time period after the first moment, the present disclosure can determine whether the foreground object corresponding to the foreground image stays in the target area for more than a certain duration, so as to determine whether the foreground object is an object left in the target area. Compared with the related art solution that only detects whether the pixel points in the picture change to detect left objects, the present disclosure can avoid the problem that the detection of foreground objects is deviated due to the sudden change of pixel points in the area caused by factors such as light change, and improves the accuracy of left object detection.

[0129] Next, in combination with the above S202, the process of the detection device for left objects determining whether there is a foreground image in the first image to be detected will be specifically introduced.

[0130] As a possible embodiment of the present disclosure, in combination with Figure 2 , as Figure 5 shown, the above S202 further includes the following S501-S502b:

[0131] S501. The detection device for left objects detects whether one or more sub-models corresponding to the second pixel points of each first pixel point of the first image to be detected match.

[0132] Among them, the foreground image determination model includes one or more sub-models corresponding to the position of each pixel point in the background image of the target area. The background image does not include the foreground image. The second pixel point is the pixel point corresponding to the position of the first pixel point in the background image.

[0133] The foreground image determination model can be a probability distribution model. For example, the probability distribution model includes a Gaussian mixture model, a single-Gaussian model (SGM). The present disclosure will be specifically described below taking the Gaussian mixture model as the foreground image determination model as an example.

[0134] Exemplarily, the Gaussian mixture model is as follows:

[0135]

[0136]

[0137] where p(x) is the Gaussian mixture model, x is any pixel in the background image of the target region, K is the number of Gaussian distribution models corresponding to the pixel x, and π k is the weight value of the k-th Gaussian distribution model, and N(x|μ k , ∑ k ) is the k-th Gaussian distribution model. The parameters of the Gaussian distribution model include μ k and ∑ k . μ k is the mean in the k-th Gaussian distribution model, and ∑ k is the covariance in the k-th Gaussian distribution model. When the pixel is represented by one-dimensional data (e.g., the pixel is represented by a pixel value), ∑ k is the variance in the k-th Gaussian distribution model. The sum of the weight values of the K Gaussian distribution models corresponding to the pixel x is 1.

[0138] The detection device for the remaining object can also determine one or more sub-models corresponding to the position of each pixel in the background image of the target region according to the weight value of each sub-model.

[0139] Specifically, the K sub-models corresponding to the pixel x are arranged in descending order of weight value. The weight values are added in sequence so that the sum of the weight values is greater than or equal to the weight threshold and the number of required sub-models is the least. The one or more sub-models required for adding the weight values are used as one or more sub-models corresponding to the position of each pixel in the background image of the target region.

[0140] Taking the sub-model as the Gaussian distribution model as an example, the number of one or more sub-models satisfies the following formula:

[0141]

[0142] where B is the number of one or more sub-models, π b is the b-th Gaussian distribution model, and T0 is the weight threshold.

[0143] Exemplarily, the weight threshold is 0.7, and the pixel 1 corresponds to 5 Gaussian distribution models. The weight value of the 1st Gaussian distribution model is 0.5, the weight value of the 2nd Gaussian distribution model is 0.2, the weight value of the 3rd Gaussian distribution model is 0.15, the weight value of the 4th Gaussian distribution model is 0.1, and the weight value of the 5th Gaussian distribution model is 0.05.

[0144] It can be obtained therefrom that the sum of the weight values of the first Gaussian distribution model and the weight values of the second Gaussian distribution model is equal to the weight threshold of 0.7, and the number of Gaussian distribution models required is the least, that is, the number is 2. Therefore, the detection device for the left-behind object determines the first Gaussian distribution model and the second Gaussian distribution model as one or more sub-models corresponding to the position where the pixel point 1 is located in the background image of the target area.

[0145] In a possible implementation manner, the detection device for the left-behind object may determine the parameter value of any first pixel point of the first image to be detected and the parameter intervals of each of the one or more sub-models of the second pixel point corresponding to the first pixel point.

[0146] When the parameter value of the first pixel point is within the parameter interval of the first sub-model, the detection device for the left-behind object determines that the first pixel point matches the first sub-model.

[0147] Wherein, the first sub-model is at least one of the one or more sub-models of the second pixel point corresponding to the first pixel point.

[0148] When the parameter value of the first pixel point is outside the parameter interval of the first sub-model, the detection device for the left-behind object determines that the first pixel point does not match the first sub-model.

[0149] Specifically, the parameter value of the first pixel point may be represented by a pixel value, a color, or a grayscale value. The parameter interval of the sub-model may be determined according to the parameters of the sub-model. For example, the lower limit of the parameter interval of the sub-model may be the difference between the mean value and a preset multiple of the standard deviation, and the upper limit of the parameter interval may be the sum of the mean value and a preset multiple of the standard deviation.

[0150] Exemplarily, the pixel value of the first pixel point is 100, and the second pixel point corresponds to 3 sub-models. Each sub-model includes two parameters, namely a mean value and a standard deviation. The mean value of the first sub-model is 90, and the standard deviation is 10. The mean value of the second sub-model is 150, and the standard deviation is 15. The mean value of the third sub-model is 200, and the standard deviation is 10. The preset multiple set in the foreground image determination model is 2.5. At this time, the parameter interval corresponding to the first sub-model is [65, 115], the parameter interval corresponding to the second sub-model is [112.5, 187.5], and the parameter interval corresponding to the third sub-model is [175, 225]. Therefore, the first pixel point matches the first sub-model and does not match the second sub-model and the third sub-model.

[0151] When at least one first pixel point does not match any of the one or more sub-models of the corresponding second pixel point, at this time, the at least one first pixel point is a foreground pixel point, and the detection device for the left-behind object may execute S502a.

[0152] Exemplarily, the first image to be detected includes a first pixel point A1 and a first pixel point B1. The background image includes a second pixel point A2 and a second pixel point B2. Among them, the first pixel point A1 corresponds to the second pixel point A2, and the first pixel point B1 corresponds to the second pixel point B2. Among them, the second pixel point A2 corresponds to sub-models X and Y. The second pixel point B2 corresponds to sub-models M and N.

[0153] When the first pixel point A1 does not match either sub-model X or sub-model Y, and the first pixel point B1 does not match either sub-model M or sub-model N, there is a foreground image in the first image to be detected.

[0154] When the first pixel point A1 does not match either sub-model X or sub-model Y, and the first pixel point B1 matches at least one of sub-models M and N, there is a foreground image in the first image to be detected.

[0155] When the first pixel point A1 matches at least one of sub-models X and Y, and the first pixel point B1 does not match either sub-model M or sub-model N, there is a foreground image in the first image to be detected.

[0156] When all the first pixel points match the sub-models of the corresponding second pixel points, each first pixel point in the first image to be detected at this time is a background pixel point, and the detection device for the remaining object can execute S502b.

[0157] Combined with the above example, when the first pixel point A1 matches at least one of sub-models X and Y, and the first pixel point B1 matches at least one of sub-models M and N, there is no foreground image in the first image to be detected.

[0158] S502a. The detection device for the remaining object determines that there is a foreground image in the first image to be detected.

[0159] S502b. The detection device for the remaining object determines that there is no foreground image in the first image to be detected.

[0160] Based on the above technical solution, the foreground image determination model in the present disclosure includes one or more sub-models corresponding to the positions of each pixel point in the background image of the target area. Therefore, this foreground image determination model can be used to represent the background image of the target area. The detection device for the remaining object determines the matching relationship between each first pixel point in the first image to be detected and the sub-models included in the foreground image determination model based on the foreground image determination model, and then can determine whether there is a foreground image in the first image to be detected.

[0161] In a possible implementation, the detection device for the legacy object can also determine a foreground image determination model based on multiple background images.

[0162] Further, the detection device for the legacy object can train a foreground image determination model according to multiple background images and a preset algorithm.

[0163] Taking the foreground image determination model as a Gaussian mixture model as an example below, the training of the foreground image determination model by the detection device for the legacy object in the present disclosure will be specifically described.

[0164] The detection device for the legacy object generates a corresponding Gaussian distribution model for each pixel point in the background image and sets its weight value to an initial weight value (for example, 1 / K) until each pixel point in the background image corresponds to K Gaussian distribution models. K is a positive integer.

[0165] The detection device for the legacy object obtains other images in the background image and, for each pixel point in the background image, determines whether there is a Gaussian distribution model that matches the pixel point among the K Gaussian distribution models corresponding to the position of the pixel point.

[0166] In the case where there is a matching Gaussian distribution model, increase the weight value of the Gaussian distribution model.

[0167] In the case where there is no matching Gaussian distribution model, generate a new Gaussian distribution model according to the pixel point and replace the Gaussian distribution model with the smallest weight value among the original K Gaussian distribution models with the new Gaussian distribution model.

[0168] The mean of the new Gaussian distribution model can be the pixel value of the pixel point. The weight and variance of the new Gaussian distribution model can be set according to the actual situation. For example, the weight value is set to the lowest value among the K Gaussian distribution models, and the variance is set to the highest value among the K Gaussian distribution models. The present disclosure does not limit this.

[0169] In a possible implementation, the detection device for the legacy object performs data processing on the weight values of the generated K Gaussian distribution models so that the sum of the weight values of the K Gaussian distribution models is 1.

[0170] Next, in combination with the above S203, the process of the detection device for the legacy object obtaining at least one tracking result will be specifically introduced.

[0171] As a possible embodiment of the present disclosure, in combination with Figure 2 or Figure 5 , as Figure 6 shown, the above S203 further includes the following S601 - S603:

[0172] The detection device for the left-behind object inputs the foreground image into the first neural network model to obtain the first image feature of the foreground image.

[0173] Among them, the first image feature can represent the image information of the foreground image. For example, the first image feature may include color features, texture features, scale features, etc. of the foreground image. The first image feature can be represented in the form of a feature vector. For example, the first image feature is [0.256314, 0.125647, 0.15248…0.1524669].

[0174] Among them, the preset tracking model includes at least one neural network model. The first neural network model is any one of the at least one neural network models. Any one of the at least one neural network models has a weight value.

[0175] When the preset tracking model includes one neural network model, the detection device for the left-behind object inputs the foreground image into the neural network model to obtain the first image feature of the foreground image.

[0176] When the preset tracking model includes multiple neural network models, the detection device for the left-behind object inputs the foreground image into the first neural network model among the multiple neural network models to obtain the first image feature of the foreground image. The algorithms of the multiple neural network models can also be the same algorithm or different algorithms.

[0177] In a possible implementation, the preset tracking model can be a siamese network model. The first neural network model and the second neural network model in the siamese network model are the same neural network model. For example, the siamese network model includes SiamRPN++, SiamRPN, SiamFC, SiamMask, etc. The preset tracking model can also be a pseudo siamese network. The first neural network model and the second neural network model in the pseudo siamese network model are different neural network models.

[0178] The neural network model can be a deep learning network, such as a convolutional neural network (CNN), a long short term memory network (LSTM).

[0179] S602. The detection device for the left-behind object inputs each of at least one comparison image into the second neural network model to obtain the second image feature corresponding to each comparison image in the at least one comparison image.

[0180] Among them, the second neural network model is any neural network model other than the first neural network model in at least one neural network model. The first neural network model and the second neural network model can be the same neural network model, or the first neural network model and the second neural network model can be different neural network models. The following is discussed in Case 1 and Case 2 respectively:

[0181] Case 1: The first neural network model and the second neural network model can be the same neural network model. After the detection device for the remaining object executes S601 above and inputs the foreground image into the neural network model to obtain the first image feature of the foreground image, the detection device for the remaining object can then execute S602 to input each of at least one comparison image into the neural network model to obtain the second image feature corresponding to each comparison image in the at least one comparison image. The detection device for the remaining object can also execute S602 first and then execute S601. The present disclosure does not limit this.

[0182] Case 2: The first neural network model and the second neural network model can also be different neural network models. Among them, the weight values of the first neural network model and the second neural network model are different, or the neural network algorithms of the first neural network model and the second neural network model are different. At this time, the detection device for the remaining object can execute S601 first and then execute S602, the detection device for the remaining object can execute S602 first and then execute S601, and the detection device for the remaining object can also perform parallel operations to execute S601 and S602 simultaneously. The present disclosure does not limit this.

[0183] In the present disclosure, the weight values of the first neural network model and the second neural network model can be obtained by training with the images of the target region, or can be obtained by training with other image data sets. The present disclosure does not limit this.

[0184] From the above two cases, it can be seen that the weight values of the first neural network model and the second neural network model in the present disclosure can be the same. The weight values of the first neural network model and the second neural network model can also be different.

[0185] In addition, each of the at least one comparison images input by the detection device for the remaining object in the present disclosure can be the comparison image itself, or can be a sub-image corresponding to the position of the foreground image in the comparison image.

[0186] Among them, the sub-image corresponding to the position of the foreground image in the comparison image can be an image of the same size as the foreground image, or can be an image of a different size from the foreground image.

[0187] Exemplarily, the area of the sub-image corresponding to the position of the foreground image in the comparison image is 2.5 times the area of the foreground image.

[0188] It should be noted that at least one comparison image includes a first comparison image, and the first comparison image is a comparison image within a preset time period in the second time period. The preset time period may be the second time period or a part of the second time period.

[0189] The detection device for the left-behind object inputs each image in the first comparison image into the second neural network model to obtain second image features corresponding to each comparison image in the first comparison image.

[0190] S603. The detection device for the left-behind object compares the first image features with the second image features corresponding to each comparison image to obtain at least one tracking result.

[0191] Among them, at least one tracking result includes at least one of a first parameter value, a range of the tracking image, and a tracking position. The preset tracking model also includes a preset loss function.

[0192] In a possible implementation manner, the detection device for the left-behind object may input the first image features and the second image features corresponding to each comparison image into the preset loss function to obtain at least one tracking result.

[0193] Exemplarily, at least one comparison image includes comparison image 1, comparison image 2, and comparison image 3. Comparison image 1 corresponds to second image feature 1, comparison image 2 corresponds to second image feature 2, and comparison image 3 corresponds to second image feature 3. The detection device for the left-behind object compares the first image features with second image feature 1 to obtain tracking results 1 and 2 corresponding to comparison image 1. The detection device for the left-behind object compares the first image features with second image feature 2 to obtain tracking results 3, 4, and 5 corresponding to comparison image 2. The detection device for the left-behind object compares the first image features with second image feature 3 to obtain tracking result 6 corresponding to comparison image 1.

[0194] Based on the above technical solutions, the detection device for the left-behind object in the present disclosure can input the foreground image into the first neural network model to obtain first image features, and input each image in at least one comparison image into the second neural network model to obtain second image features, so as to obtain at least one tracking result according to the first image features and the second image features. In this way, the detection device for the left-behind object can track the foreground object corresponding to the foreground image in the comparison image based on the at least one tracking result, so as to further determine whether there is a left-behind object in the target area, improving the accuracy of left-behind object detection.

[0195] The following takes the preset tracking model as an example of a Siamese network model to specifically describe the method for detecting left-behind objects involved in the present disclosure. As Figure 7As shown in the figure, the Siamese network model includes a neural network model 1, a neural network model 2, and a loss function. Among them, the weight values of the neural network model 1 and the neural network model 2 are the same.

[0196] The detection device for the left-behind object inputs the foreground image as a template image into the neural network model 1 to obtain the first image feature of the foreground image. The detection device for the left-behind object inputs each image in at least one comparison image as a search regression image into the neural network model 2 to obtain the second image feature corresponding to each comparison image in the at least one comparison image. The detection device for the left-behind object inputs the obtained first image feature and second image feature into the loss function to obtain at least one tracking result.

[0197] Next, in combination with the above S204, the process of the detection device for the left-behind object obtaining at least one tracking result will be specifically introduced.

[0198] As a possible embodiment of the present disclosure, in combination with Figure 6 , as Figure 8 shown, the above S204 further includes the following S801-S802:

[0199] S801. When at least one tracking result meets the second preset condition, the detection device for the left-behind object determines that the foreground object is an object left in the target area.

[0200] Among them, the second preset condition includes at least one limiting condition, and the at least one limiting condition corresponds to the at least one tracking result.

[0201] The at least one limiting condition includes at least one of a first parameter value being greater than a third threshold, the range of the tracking image being greater than a fourth threshold, and the tracking position being within at least one preset range in the comparison image. The third threshold is the similarity threshold between the foreground image and the tracking image, and the fourth threshold is the range threshold of the tracking image.

[0202] The above S801 can be implemented as: when the first parameter value is greater than the third threshold, and / or the range of the tracking image is greater than the fourth threshold, and / or the tracking position is within at least one preset range in the comparison image, the detection device for the left-behind object determines that the foreground object is an object left in the target area.

[0203] It should be noted that in the present disclosure, the first parameter value is used to represent the similarity between the foreground image and the corresponding tracking image in the comparison image. The higher the first parameter value, the higher the similarity between the foreground image and the tracking image. Therefore, when the first parameter value is greater than the third threshold, the detection device for the left-behind object can determine that the object corresponding to the tracking image is the foreground object corresponding to the foreground image.

[0204] In the present disclosure, the range of the tracking image is the ratio of the area of the tracking image to the area of the comparison image or the number of pixel points of the tracking image, which is used to represent the size of the object corresponding to the tracking image. Therefore, by determining whether the range of the tracking image is greater than the fourth threshold, the detection device for the remaining object can determine the size change of the object corresponding to the tracking image, so as to judge whether the object will have an impact on the target area.

[0205] Exemplarily, the object is an inflatable balloon. After the detection device for the remaining object detects the foreground image corresponding to the inflatable balloon, the inflatable balloon gradually deflates (that is, the area of the foreground image gradually becomes smaller). At this time, the deflated balloon cannot cause a great impact on the target area. The detection device for the remaining object can determine the range of the tracking image according to the deflated balloon in the comparison image, and then determine that the range of the tracking image does not meet the second preset condition.

[0206] In the present disclosure, the detection device for the remaining object can also set at least one preset range in the target area, and then judge whether the tracking position of the tracking image is within the at least one preset range in the comparison image, so as to realize the targeted detection of the key area in the target area.

[0207] S802. When any one of the at least one tracking result does not meet the second preset condition, the detection device for the remaining object determines that the foreground object is not the object remaining in the target area.

[0208] For the specific judgment process, reference can be made to S801 above, which will not be elaborated here.

[0209] Based on the above technical solution, the detection device for the remaining object in the present disclosure can detect whether the foreground object is the object remaining in the target area by judging whether at least one tracking result meets the second preset condition. Among them, the at least one tracking result includes at least one of the first parameter value, the range of the tracking image, and the tracking position. The first parameter value can represent the similarity between the foreground image and the tracking image. The range of the tracking image can represent the size change of the object corresponding to the tracking image. The tracking position can represent the position movement of the object corresponding to the tracking image. Therefore, the detection device for the remaining object can detect the remaining situation of the foreground object based on at least one of the similarity, size change, and position movement of the tracking image, improving the accuracy of the detection of the remaining object.

[0210] As a possible embodiment of the present disclosure, in combination with Figure 2 , as Figure 9 shown, before S203, the method further includes the following S901-S903:

[0211] S901. The detection device for the left-behind object acquires one or more background images and determines a sub-background image for each of the one or more background images.

[0212] Among them, the position of the sub-background image in the background image corresponds to the position of the foreground image in the first image to be detected. The range of the sub-background image may be the same as that of the foreground image. The range of the sub-background image may also be different from that of the foreground image. The range of the sub-background image is the area occupied by the sub-background image in the background image.

[0213] It should be noted that the one or more background images acquired by the detection device for the left-behind object are images of the target area before the first moment, that is, images of the target area before the detection device for the left-behind object determines that there is a foreground image in the first image to be detected.

[0214] Therefore, the one or more background images are images determined by the detection device for the left-behind object according to the foreground image determination model as images where there is no foreground image, or images that do not meet the first preset condition.

[0215] S902. The detection device for the left-behind object inputs the foreground image and one or more sub-background images into the verification model to obtain at least one second parameter value.

[0216] Among them, the verification model is used to calculate the similarity between the foreground image and one or more sub-background images. The second parameter value is used to represent the similarity between the foreground image and the sub-background image. The at least one second parameter value is the second parameter value of one or more sub-background images.

[0217] It should be noted that the algorithm of the verification model in this disclosure is different from the algorithm of the above foreground image determination model. For example, the verification model may be the preset tracking model in the above method, or other models for calculating image similarity.

[0218] If the verification model is the preset tracking model in the above method, the specific process for the left-behind object to obtain at least one second parameter value can refer to the process in the above method where the left-behind object obtains the first parameter value according to the foreground image and the comparison image, which will not be elaborated here.

[0219] S903. When at least one second parameter value is less than the fifth threshold, the detection device for the left-behind object determines that there is a foreground image in the first image to be detected.

[0220] Among them, the fifth threshold is the similarity threshold between the foreground image and the sub-background image, which can be specifically set according to the actual situation, and this disclosure does not limit it.

[0221] It should be noted that the sub-background image is a sub-image in the background image of the target area before the first moment. Therefore, when at least one second parameter value is less than the fifth threshold, it indicates that before the foreground image determination model determines that there is a foreground image in the first image to be detected, the similarity between the sub-background image of the background image of the target area and the foreground image is low, that is, the sub-background image is different from the foreground image. That is to say, the object corresponding to the foreground image determined by the foreground image determination model of the left-behind object detection device did not appear at the corresponding position of the target area before the first moment. On the contrary, when at least one second parameter value is greater than or equal to the fifth threshold, it indicates that the object corresponding to the foreground image determined by the foreground image determination model of the left-behind object detection device was already located at the corresponding position of the target area before the first moment.

[0222] Among them, at least one second parameter value being less than the fifth threshold may mean that any one of the at least one second parameter values is less than the fifth threshold, or it may mean that the average value of the at least one second parameter values is less than the fifth threshold.

[0223] Exemplarily, as Figure 10 shown, Figure 10 a and b in Figure 10 are the background images of the target area before the first moment, Figure 10 c in Figure 10 is the first image to be detected at the first moment. Due to lighting factors, the foreground image determination model of the left-behind object detection device determines that there is a foreground image 103 in the first image to be detected. The left-behind object detection device obtains Figure 10 a and b in Figure 10 and determines the sub-background image 101 in Figure 10 a and the sub-background image 102 in Figure 10 b. The left-behind object detection device inputs the foreground image 103, the sub-background image 101, and the sub-background image 102 into the verification model, and obtains the second parameter value 1 corresponding to the sub-background image 101 and the second parameter value 2 corresponding to the sub-background image 102. Among them, the second parameter value 1 is 0.7, the second parameter value 2 is 0.8, and the fifth threshold is 0.5. Since there is a parameter value greater than or equal to the fifth threshold among the second parameter value 1 and the second parameter value 2, the similarity between the foreground image and the sub-image at the corresponding position before the first moment is high, and the left-behind object detection device determines that there is no foreground image in the first image to be detected.

[0224] On the contrary, the process of the left-behind object detection device determining that there is a foreground image in the first image to be detected will not be elaborated here.

[0225] Based on the above technical solution, before the detection device for the left-behind object in the present disclosure tracks the foreground object according to the foreground image and at least one comparison image, the verification model can further verify the foreground image determined by the foreground image determination model according to the foreground image and at least one sub-background image before the first moment, and determine whether the foreground image has appeared before the first moment. If the similarity between the foreground image and at least one sub-background image before the first moment is low, it indicates that the foreground image has not appeared before the first moment, that is, there is a foreground image in the first image to be detected. Otherwise, it indicates that the foreground image has appeared before the first moment, that is, there is no foreground image in the first image to be detected. In this way, the detection device for the left-behind object can first verify whether the foreground image determined by the foreground image determination model is accurate through the verification model before performing the target tracking operation, thereby improving the detection efficiency and accuracy of the left-behind object detection.

[0226] As a possible embodiment of the present disclosure, after the detection device for the left-behind object in S204 detects whether the foreground object is an object left in the target area, the method further includes:

[0227] When the foreground object is an object left in the target area, the detection device for the left-behind object outputs a prompt message.

[0228] Among them, the prompt message is used to indicate that there is a left-behind object in the target area. The prompt message can be a text prompt message, a voice prompt message, or an image prompt message.

[0229] Hereinafter, in combination with the above embodiments, when there is no foreground image in the first image to be detected, or the foreground image does not meet the first preset condition, or the foreground object is not an object left in the target area, the detection device for the left-behind object can update the foreground image determination model.

[0230] As a possible embodiment of the present disclosure, as Figure 11 shown, the method by which the detection device for the left-behind object can update the foreground image determination model may include:

[0231] S1101. The detection device for the left-behind object determines an update image.

[0232] Among them, the update image is the first image to be detected.

[0233] It should be noted that when the foreground image does not meet the first preset condition, or the foreground object is not an object left in the target area, the update image can be the first image to be detected or the image of the target area at the current moment.

[0234] S1102. The detection device for the left-behind object updates the foreground image determination model according to the updated image to obtain the updated foreground image determination model.

[0235] Among them, the updated foreground image determination model can be used to determine whether there is a foreground image in the image.

[0236] In a possible implementation manner, the detection device for the left-behind object can also retrain according to a preset algorithm and the updated image to obtain the updated foreground image determination model. The specific training process can refer to the above process of training the model, which will not be elaborated here.

[0237] In another possible implementation manner, the detection device for the left-behind object can update the sub-model in the foreground image determination model before update according to the updated image based on the foreground image determination model before update to obtain the updated foreground image determination model.

[0238] The specific process of this implementation manner can be as follows:

[0239] 1. For each third pixel point in the updated image, the detection device for the left-behind object detects whether there is a sub-model in one or more sub-models corresponding to the second pixel point that matches the third pixel point.

[0240] Among them, the foreground image determination model includes one or more sub-models corresponding to the position of each pixel point in the background image of the target area, each sub-model corresponds to a weight value, and the second pixel point is the pixel point in the background image corresponding to the position of the third pixel point.

[0241] The method for detecting whether the third pixel point matches the sub-model can refer to the relevant description of S501 above, which will not be elaborated here.

[0242] 2. In the case where there is a second sub-model in one or more sub-models, the detection device for the left-behind object increases the weight value of the second sub-model and decreases the weight value of the third sub-model.

[0243] Among them, the second sub-model is the sub-model in one or more sub-models that matches the third pixel point, and the third sub-model is the other sub-models in one or more sub-models except the second sub-model. The sub-model in one or more sub-models corresponding to the second pixel point that matches the third pixel point can be one sub-model or multiple sub-models.

[0244] It should be noted that the higher the weight value of the sub-model, the greater the proportion of the sub-model in one or more sub-models. Therefore, when there is a second sub-model that matches the third pixel point in one or more sub-models, the weight value of the second sub-model can be increased and the weight value of the third sub-model can be decreased, so as to increase the proportion of the second sub-model.

[0245] The detection device for the left-behind object obtains an updated foreground image determination model according to the increased weight value of the second sub-model and the decreased weight value of the third sub-model.

[0246] Among them, the updated foreground image determination model includes the second sub-model with the increased weight value corresponding to each third pixel point in the updated image and the third sub-model with the decreased weight.

[0247] 3. In the case where the second sub-model does not exist in one or more sub-models, the detection device for the left-behind object generates a fourth sub-model according to the third pixel point, and replaces the sub-model with the smallest weight value in the one or more sub-models with the fourth sub-model, so as to obtain an updated foreground image determination model.

[0248] It should be noted that when the second sub-model does not exist in one or more sub-models, it means that the third pixel point is determined as a foreground pixel point in the foreground image determination model. Therefore, the detection device for the left-behind object can generate a fourth sub-model according to the third pixel point, and replace the sub-model with the smallest weight value in the one or more sub-models with the fourth sub-model, so that the updated foreground image determination model can determine that the third pixel point is a background pixel point.

[0249] Based on the above technical solutions, when there is no foreground image in the first image to be detected, or the foreground image does not meet the first preset condition, or the foreground object is not an object left in the target area, the detection device for the left-behind object can update the foreground image determination model according to the updated image to obtain an updated foreground image determination model, thereby avoiding the problem that the foreground image detects other pixel points in the target area except the foreground pixel points corresponding to the left-behind object as foreground pixel points, and improving the accuracy of left-behind object detection.

[0250] As a possible embodiment of the present disclosure, the method provided in the embodiments of the present application may include: the detection device for the left-behind object re-detects whether there is a left-behind object in the target area.

[0251] As Figure 12 shown, the specific process of the detection device for the left-behind object re-detecting whether there is a left-behind object in the target area may include S1201-S1204:

[0252] S1201. The detection device for the left-behind object obtains a second image to be detected of the target area at the second moment.

[0253] Among them, the second moment is the moment after the first moment. For specific reference, see the relevant description in S201, which will not be elaborated here.

[0254] S1202. The detection device for the left-behind object determines whether there is a foreground image in the second image to be detected according to the foreground image determination model.

[0255] Among them, the foreground image determination model can be the foreground image determination model before update or the foreground image determination model after update.

[0256] For specific reference, see the relevant description in S202, which will not be elaborated here.

[0257] S1203. When there is a foreground image in the second image to be detected and the foreground image meets the first preset condition, the detection device for the remaining object inputs the foreground image and at least one second comparison image into a preset tracking model to obtain at least one tracking result.

[0258] For specific reference, see the relevant description in S203, which will not be elaborated here.

[0259] S1204. The detection device for the remaining object detects whether the foreground object is an object left in the target area according to at least one tracking result.

[0260] For specific reference, see the relevant description in S204, which will not be elaborated here.

[0261] Based on the above technical solution, the detection device for the remaining object can dynamically detect whether there is a remaining object in the target area, and the detection method is flexible and convenient.

[0262] Embodiments of the present disclosure can divide the detection device for the remaining object into functional modules or functional units according to the above method examples. For example, each functional module or functional unit can be corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware, or in the form of a software functional module or functional unit. Among them, the division of modules or units in the embodiments of the present disclosure is illustrative, only a logical function division, and there can be other division methods in actual implementation.

[0263] Such as Figure 13 shown, is a schematic structural diagram of a detection device 130 for a remaining object provided according to some embodiments. The device includes:

[0264] A processing unit 1301, configured to obtain a first image to be detected of the target area at a first moment.

[0265] Among them, the detection device 130 for the remaining object may further include a communication unit 1302.

[0266] The detection device 130 for the remaining object can receive the first image to be detected of the target area in real time through the communication unit 1302, and obtain the first image to be detected of the target area at the first moment therefrom through the processing unit 1301.

[0267] The detection device 130 for the remaining object can also periodically receive the first image to be detected of the target area through the communication unit 1302, and obtain the first image to be detected of the target area at the first moment from it through the processing unit 1301.

[0268] The processing unit 1301 is further configured to determine whether there is a foreground image in the first image to be detected according to the foreground image determination model.

[0269] Wherein, the foreground image is an image corresponding to the foreground object in the target area, and the foreground image determination model is used to determine whether there is a foreground image in the image.

[0270] When there is a foreground image in the first image to be detected and the foreground image meets the first preset condition, the processing unit 1301 is further configured to input the foreground image and at least one comparison image into a preset tracking model to obtain at least one tracking result.

[0271] Wherein, the at least one comparison image is an image of the target area in the second time period, the second time period is the time period after the first moment, and the first preset condition includes at least one of the ratio of the area of the foreground image to the area of the first image to be detected is greater than the first threshold value, and the number of pixel points of the foreground image is greater than the second threshold value.

[0272] The processing unit 1301 is further configured to detect whether the foreground object is an object left in the target area according to the at least one tracking result.

[0273] In some embodiments, the foreground image determination model includes one or more sub-models corresponding to the position of each pixel point in the background image of the target area, the background image does not include the foreground image, and the processing unit 1301 is configured to: detect whether one or more sub-models of the second pixel points corresponding to each first pixel point of the first image to be detected match, the second pixel point is the pixel point corresponding to the position of the first pixel point in the background image; when one or more sub-models of at least one first pixel point and the corresponding second pixel points do not match, determine that there is a foreground image in the first image to be detected; when the sub-models of all first pixel points and the corresponding second pixel points match, determine that there is no foreground image in the first image to be detected.

[0274] In some embodiments, the processing unit 1301 is configured to: determine the parameter value of any first pixel point of the first image to be detected and the parameter intervals of each sub-model among one or more sub-models of the second pixel point corresponding to the first pixel point; determine that the first pixel point matches the first sub-model when the parameter value of the first pixel point is within the parameter interval of the first sub-model, where the first sub-model is at least one sub-model among one or more sub-models of the second pixel point corresponding to the first pixel point; and determine that the first pixel point does not match the first sub-model when the parameter value of the first pixel point is outside the parameter interval of the first sub-model.

[0275] In some embodiments, the preset tracking model includes at least one neural network model; the processing unit 1301 is configured to: input the foreground image into the first neural network model to obtain the first image feature of the foreground image, where the first neural network model is any neural network model among at least one neural network model; input each of at least one comparison image into the second neural network model to obtain the second image feature corresponding to each comparison image in the at least one comparison image, where the second neural network model is any neural network model other than the first neural network among at least one neural network model; and compare the first image feature and the second image feature corresponding to each comparison image to obtain at least one tracking result.

[0276] In some embodiments, at least one tracking result includes at least one of a first parameter value, the range of the tracking image, and the tracking position; where the tracking image is the sub-image with the highest similarity to the foreground image in the comparison image, the first parameter value is used to represent the similarity between the foreground image and the tracking image, the range of the tracking image is the area occupied by the tracking image in the comparison image, and the tracking position is the position of the tracking image in the comparison image.

[0277] In some embodiments, the processing unit 1301 is configured to: determine that the foreground object is an object left in the target area when at least one tracking result meets the second preset condition, where the second preset condition includes at least one limiting condition, and at least one limiting condition corresponds to at least one tracking result; and determine that the foreground object is not an object left in the target area when any one of the at least one tracking result does not meet the second preset condition.

[0278] In some embodiments, the processing unit 1301 is configured to: determine that the foreground object is an object left in the target area when the first parameter value is greater than a third threshold, and / or the range of the tracking image is greater than a fourth threshold, and / or the tracking position is within at least one preset range in the comparison image.

[0279] In some embodiments, the processing unit 1301 is further configured to: obtain one or more background images, and determine sub-background images of each of the one or more background images, where the positions of the sub-background images in the background images correspond to the position of the foreground image in the first image to be detected; input the foreground image and the one or more sub-background images into a verification model to obtain at least one second parameter value; the second parameter value is used to represent the similarity between the foreground image and the sub-background images; in the case where at least one second parameter value is less than a fifth threshold, determine that there is a foreground image in the first image to be detected.

[0280] In some embodiments, in the case where there is no foreground image in the first image to be detected, or the foreground image does not meet the first preset condition, or the foreground object is not an object remaining in the target area, the processing unit 1301 is further configured to update the foreground image determination model.

[0281] In some embodiments, the processing unit 1301 is configured to: determine an updated image; the updated image is the first image to be detected; update the foreground image determination model according to the updated image.

[0282] In some embodiments, the foreground image determination model includes one or more sub-models corresponding to the positions of each pixel point in the background image of the target area, and each sub-model corresponds to a weight value; the processing unit 1301 is configured to: for each third pixel point in the updated image, detect whether there is a sub-model in the one or more sub-models corresponding to the second pixel point that matches the third pixel point, where the second pixel point is the pixel point in the background image corresponding to the position of the third pixel point; in the case where there is a second sub-model in the one or more sub-models, increase the weight value of the second sub-model and decrease the weight value of the third sub-model, where the second sub-model is the sub-model in the one or more sub-models that matches the third pixel point, and the third sub-model is the other sub-models in the one or more sub-models except the second sub-model; update the foreground image determination model according to the increased weight value of the second sub-model and the decreased weight value of the third sub-model.

[0283] In some embodiments, the processing unit 1301 is configured to: in the case where there is no second sub-model in the one or more sub-models, generate a fourth sub-model according to the third pixel point, and replace the sub-model with the smallest weight value in the one or more sub-models with the fourth sub-model to obtain an updated foreground image determination model.

[0284] In some embodiments, the processing unit 1301 is configured to: obtain a second image to be detected of the target area at a second moment; the second moment is a moment after the first moment; determine whether there is a foreground image in the second image to be detected according to the foreground image determination model; in the case that there is a foreground image in the second image to be detected and the foreground image meets a first preset condition, input the foreground image and at least one second comparison image into a preset tracking model to obtain at least one tracking result; the at least one second comparison image is an image of the target area within a third time period, the third time period is a time period after the second moment, and the first preset condition includes at least one of a ratio of the area of the foreground image to the area of the second image to be detected being greater than a first threshold value and the number of pixel points of the foreground image being greater than a second threshold value; detect whether the foreground object is an object left in the target area according to the at least one tracking result.

[0285] In some embodiments, the communication unit 1302 is configured to: output a prompt message in the case that the foreground object is an object left in the target area.

[0286] When implemented by hardware, the communication unit 1302 in the embodiments of the present disclosure may be integrated on a communication interface, and the processing unit 1301 may be integrated on a processor. The specific implementation manner is as Figure 14 shown.

[0287] Figure 14 Fig. shows another possible structural schematic diagram of the detection device for the left-behind object involved in the above embodiments. The detection device 140 for the left-behind object includes: a processor 1402 and a communication interface 1403. The processor 1402 is configured to control and manage the actions of the detection device 140 for the left-behind object. For example, execute the steps executed by the above-mentioned processing unit 1301, and / or be configured to execute other processes of the technologies described herein. The communication interface 1403 is configured to support the communication of the detection device 140 for the left-behind object with other network entities. For example, execute the steps executed by the above-mentioned communication unit 1302. The detection device 140 for the left-behind object may further include a memory 1401 and a bus 1404. The memory 1401 is configured to store the program code and data of the detection device 140 for the left-behind object.

[0288] Among them, the memory 1401 may be a memory in the detection device 140 for the left-behind object, etc. This memory may include a volatile memory, such as a random access memory; this memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; this memory may further include a combination of the above types of memories.

[0289] The above-mentioned processor 1402 may implement or execute various exemplary logic blocks, modules, and circuits described in connection with the present disclosure. The processor may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in connection with the present disclosure. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0290] The bus 1404 may be an Extended Industry Standard Architecture (EISA) bus or the like. The bus 1404 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 14 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0291] Figure 14 The detection device 140 of the legacy object in the figure may also be a chip. The chip includes one or more than two (including two) processors 1402 and a communication interface 1403.

[0292] Optionally, the chip further includes a memory 1401. The memory 1401 may include a read-only memory and a random access memory, and provide operation instructions and data to the processor 1402. A part of the memory 1401 may also include a non-volatile random access memory (NVRAM).

[0293] In some embodiments, the memory 1401 stores the following elements, execution modules, or data structures, or subsets thereof, or extended sets thereof.

[0294] In the embodiments of the present disclosure, by invoking the operation instructions stored in the memory 1401 (the operation instructions may be stored in the operating system), corresponding operations are executed.

[0295] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of description, only the above division of each functional module is used as an example. In actual applications, the above functions may be allocated to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working processes of the systems, devices, and units described above may refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0296] Some embodiments of the present disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) storing computer program instructions, which, when running on a computer (e.g., a detection device for left-behind objects), cause the computer to execute the method for detecting left-behind objects described in any one of the above embodiments.

[0297] Exemplarily, the above computer-readable storage medium may include, but is not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes, etc.), optical discs (e.g., CDs (Compact Disks), DVDs (Digital Versatile Disks), etc.), smart cards, and flash memory devices (e.g., EPROMs (Erasable Programmable Read-Only Memories), cards, sticks, or key drives, etc.). The various computer-readable storage media described in the present disclosure may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0298] Some embodiments of the present disclosure further provide a computer program product, e.g., the computer program product is stored on a non-transitory computer-readable storage medium. The computer program product includes computer program instructions, which, when executed on a computer (e.g., a detection device for left-behind objects), cause the computer to execute the method for detecting left-behind objects described in the above embodiments.

[0299] Some embodiments of the present disclosure further provide a computer program. When the computer program is executed on a computer (e.g., a detection device for left-behind objects), the computer program causes the computer to execute the method for detecting left-behind objects described in the above embodiments.

[0300] The beneficial effects of the above computer-readable storage medium, computer program product, and computer program are the same as those of the method for detecting left-behind objects described in some of the above embodiments, and will not be elaborated herein.

[0301] In several embodiments provided by the present disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections between each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms.

[0302] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0303] In addition, in each embodiment of the present disclosure, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0304] As described above, it is only the specific implementation manner of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present disclosure who thinks of changes or substitutions should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A method for detecting a legacy object, characterized in that, The method includes: Obtaining a first image to be detected of a target area at a first moment; Determining whether there is a foreground image in the first image to be detected according to a foreground image determination model; the foreground image is an image corresponding to a foreground object within the target area; When there is the foreground image in the first image to be detected and the foreground image meets a first preset condition, inputting the foreground image and at least one comparison image into a preset tracking model to obtain at least one tracking result; the at least one comparison image is an image of the target area within a second time period, the second time period being a time period after the first moment, and the first preset condition includes at least one of a ratio of an area of the foreground image to an area of the first image to be detected being greater than a first threshold value and a number of pixel points of the foreground image being greater than a second threshold value; the at least one tracking result includes at least one of a first parameter value, a range of a tracking image, and a tracking position; wherein, the tracking image is a sub-image with the highest similarity to the foreground image in the comparison image, the first parameter value is used to represent a similarity between the foreground image and the tracking image, the range of the tracking image is an area occupied by the tracking image in the comparison image, and the tracking position is a position of the tracking image in the comparison image; Detecting whether the foreground object is an object left in the target area according to the at least one tracking result.

2. The method according to claim 1, wherein The foreground image determination model includes one or more sub-models corresponding to positions of each pixel point in a background image of the target area, the background image not including the foreground image, and determining whether there is a foreground image in the first image to be detected according to the foreground image determination model includes: Detecting whether one or more sub-models of a second pixel point corresponding to each first pixel point of the first image to be detected match, the second pixel point being a pixel point corresponding to the first pixel point in the background image; When one or more sub-models of at least one first pixel point and the corresponding second pixel point do not match, determining that there is the foreground image in the first image to be detected; When sub-models of all first pixel points and the corresponding second pixel points match, determining that there is no foreground image in the first image to be detected.

3. The method according to claim 2, wherein Detecting whether one or more sub-models of a second pixel point corresponding to each first pixel point of the first image to be detected match includes: Determining a parameter value of any one of the first pixel points of the first image to be detected and a parameter interval of each of the one or more sub-models of the second pixel point corresponding to the first pixel point; When the parameter value of the first pixel point is within the parameter interval of a first sub-model, determining that the first pixel point matches the first sub-model; the first sub-model being at least one of the one or more sub-models of the second pixel point corresponding to the first pixel point; In the case where the parameter value of the first pixel point is outside the parameter interval of the first sub-model, it is determined that the first pixel point does not match the first sub-model.

4. The method according to any one of claims 1 to 3, characterized in that, The preset tracking model includes at least one neural network model; the step of inputting the foreground image and at least one comparison image into the preset tracking model to obtain at least one tracking result includes: Inputting the foreground image into a first neural network model to obtain a first image feature of the foreground image, where the first neural network model is any one of the at least one neural network model; Inputting each of the at least one comparison image into a second neural network model to obtain a second image feature corresponding to each comparison image in the at least one comparison image, where the second neural network model is any one of the at least one neural network model other than the first neural network; Comparing the first image feature and the second image features corresponding to each comparison image to obtain the at least one tracking result.

5. The method according to claim 1, wherein The step of detecting whether the foreground object is an object left in the target area according to the at least one tracking result includes: In the case where the at least one tracking result meets a second preset condition, it is determined that the foreground object is an object left in the target area, where the second preset condition includes at least one limiting condition, and the at least one limiting condition corresponds to the at least one tracking result; In the case where any one of the at least one tracking result does not meet the second preset condition, it is determined that the foreground object is not an object left in the target area.

6. The method according to claim 5, wherein The step of, in the case where the at least one tracking result meets the second preset condition, determining that the foreground object is an object left in the target area includes: In the case where the first parameter value is greater than a third threshold, and / or the range of the tracking image is greater than a fourth threshold, and / or the tracking position is within at least one preset range in the comparison image, it is determined that the foreground object is an object left in the target area.

7. The method according to any one of claims 1-3, 5-6, characterized in that, Before the step of inputting the foreground image and at least one comparison image into the preset tracking model to obtain at least one tracking result, the method further includes: Obtaining one or more background images and determining a sub-background image of each background image in the one or more background images, where the position of the sub-background image in the background image corresponds to the position of the foreground image in the first image to be detected; Inputting the foreground image and one or more of the sub-background images into a verification model to obtain at least one second parameter value; the second parameter value is used to represent the similarity between the foreground image and the sub-background image; In the case where the at least one second parameter value is less than a fifth threshold, it is determined that the foreground image exists in the first image to be detected.

8. The method according to any one of claims 1-3, 5-6, characterized in that The method further includes: When the foreground image does not exist in the first image to be detected, or the foreground image does not meet the first preset condition, or the foreground object is not an object remaining in the target area, update the foreground image determination model.

9. The method according to claim 8, wherein The step of updating the foreground image determination model when the foreground image does not exist in the first image to be detected, or the foreground image does not meet the first preset condition, or the foreground object is not an object remaining in the target area, includes: Determine an updated image; the updated image is the first image to be detected; Update the foreground image determination model according to the updated image to obtain an updated foreground image determination model.

10. The method according to claim 9, characterized in that, The foreground image determination model includes one or more sub-models corresponding to the positions of each pixel point in the background image of the target area, and each sub-model corresponds to a weight value; the method further includes: For each third pixel point in the updated image, detect whether there is a sub-model in one or more sub-models corresponding to the second pixel point that matches the third pixel point, where the second pixel point is the pixel point in the background image corresponding to the position of the third pixel point; The step of updating the foreground image determination model according to the updated image to obtain an updated foreground image determination model includes: When there is a second sub-model in the one or more sub-models, increase the weight value of the second sub-model and decrease the weight value of the third sub-model, where the second sub-model is the sub-model in the one or more sub-models that matches the third pixel point, and the third sub-model is the other sub-models in the one or more sub-models except the second sub-model; Obtain the updated foreground image determination model according to the increased weight value of the second sub-model and the decreased weight value of the third sub-model.

11. The method according to claim 10, characterized in that, The step of updating the foreground image determination model according to the updated image to obtain an updated foreground image determination model includes: When there is no such second sub-model in the one or more sub-models, generate a fourth sub-model according to the third pixel point, and replace the sub-model with the smallest weight value in the one or more sub-models with the fourth sub-model to obtain the updated foreground image determination model.

12. The method according to any one of claims 1-3, 5-6, 9-11, characterized in that, The method further includes: Obtain a second image to be detected of the target area at a second time; the second time is a time after the first time; Determine whether the foreground image exists in the second image to be detected according to the foreground image determination model. When the foreground image exists in the second image to be detected and the foreground image meets the first preset condition, input the foreground image and at least one second comparison image into a preset tracking model to obtain at least one tracking result; the at least one second comparison image is an image of the target area within a third time period, and the third time period is a time period after the second moment, and the first preset condition includes at least one of a ratio of an area of the foreground image to an area of the second image to be detected being greater than a first threshold and a number of pixel points of the foreground image being greater than a second threshold; According to the at least one tracking result, detect whether the foreground object is an object left in the target area.

13. The method according to any one of claims 1-3, 5-6, 9-11, characterized in that, The method further includes: When the foreground object is an object left in the target area, output a prompt message.

14. A detecting device for a legacy object, characterized in that including: a processing unit configured to obtain an image to be detected of a target area at a first moment; the processing unit is further configured to determine whether a foreground image exists in the image to be detected according to a foreground image determination model; the foreground image is an image corresponding to a foreground object in the target area; When the foreground image exists in the image to be detected and the foreground image meets the first preset condition, the processing unit is further configured to input the foreground image and at least one comparison image into a preset tracking model to obtain at least one tracking result; the at least one comparison image is an image of the target area within a second time period, and the second time period is a time period after the first moment, and the first preset condition includes at least one of a ratio of an area of the foreground image to an area of the image to be detected being greater than a first threshold and a number of pixel points of the foreground image being greater than a second threshold; the at least one tracking result includes at least one of a first parameter value, a range of a tracking image, and a tracking position; wherein, the tracking image is a sub-image with the highest similarity to the foreground image in the comparison image, the first parameter value is used to represent the similarity between the foreground image and the tracking image, the range of the tracking image is an area occupied by the tracking image in the comparison image, and the tracking position is a position of the tracking image in the comparison image; the processing unit is further configured to detect whether the foreground object is an object left in the target area according to the at least one tracking result.

15. A detection device for a legacy object, characterized in that, including: a processor and a communication interface; the communication interface is coupled to the processor, and the processor is configured to run a computer program or instruction to implement the method for detecting a left object as described in any one of claims 1-13.

16. A detection system for a legacy object, characterized in that, including a device for detecting a left object and at least one camera device, and the device for detecting a left object is configured to execute the method for detecting a left object as described in any one of claims 1-13.

17. A non-transitory computer-readable storage medium, characterized in that, Instructions are stored in the non-transitory computer-readable storage medium, and when the computer executes the instructions, the computer executes the method for detecting a left object as described in any one of claims 1-13.

18. A computer program product, characterized in that, The computer program product includes computer program instructions which, when executed on a computer, cause the computer to perform the method for detecting a legacy object as described in any one of claims 1-13.

Images