Object movement detection method and device
By identifying obscured items and determining their movement using the difference image and preset movement prediction model, the resource waste caused by movement detection of all items in the prior art is solved, and efficient detection of items that need to be tracked is achieved.
Patent Information
- Application Number
- CN202011471372.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-14
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-12-14
AI Technical Summary
In the prior art, the movement detection of all items leads to wasting computer resources, and not all items require target tracking, resulting in excessive resource utilization.
By identifying the obstructed target item, obtain images before and after it is not obstructed, use the difference image and the preset movement prediction model to determine whether the item is moved, and avoid movement detection of all items.
It effectively avoids the problem of heavy system burden and wasting computer resources, and only performs mobile detection on items that need to be tracked, which improves resource utilization.
Smart Images

Figure CN112529959B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method and device for detecting object movement. Background Art
[0002] Object management is required in many fields, and one of the core functions of object management is to track objects. In actual situations, object tracking is generally only performed on moving objects. Therefore, before tracking an object, it is necessary to detect whether the object is moving.
[0003] However, in the prior art, all objects are continuously detected for movement, which results in the need to occupy considerable computer resources for the detection of movement of objects. Since not all objects are worth tracking, the detection of movement of all objects is undoubtedly a waste of computer resources. Summary of the invention
[0004] In view of the above problems, the present invention provides a method and device for detecting object movement that overcomes the above problems or at least partially solves the above problems. The technical solution is as follows:
[0005] A method for detecting object movement, comprising:
[0006] Identify occluded objects;
[0007] Obtaining a first object image of the obscured target object in a first image, wherein the first image is an image captured before the obscured target object is not obscured, and the first object image is located in a first area of the first image;
[0008] Obtaining a second object image of the obscured target object in the second image, wherein the second image is an image captured after the obscured target object is no longer obscured, and the second object image is located in the first area of the second image;
[0009] It is determined whether the obscured target object is moved during the obscuration period according to the first object image and the second object image.
[0010] Optionally, the identifying the obscured target object includes:
[0011] Determine a positional relationship between a first position mark of the target object in a captured frame of image and a second position mark of a person in the frame of image, and when the positional relationship is a preset positional relationship, determine that the target object in the frame of image is blocked by the person.
[0012] Optionally, the first position marker and the second position marker are both circumscribed rectangular frames, and the preset position relationship is: the first position marker and the second position marker have an intersection.
[0013] Optionally, the image used to identify the obscured target object and the first image are in the same image sequence, and the images in the image sequence are arranged in chronological order according to the acquisition time; when the obscured target object is identified from multiple consecutive frames of images, the first image is the previous frame image of the first frame image in the multiple consecutive frames of images.
[0014] Optionally, the image used to identify the obscured target object and the second image are located in the same image sequence, and the images in the image sequence are arranged in chronological order according to the acquisition time; when the obscured target object is identified from multiple consecutive frames of images, the second image is the frame image after the last frame image in the multiple consecutive frames of images.
[0015] Optionally, determining whether the obscured target object is moved during the obscuration period according to the first object image and the second object image includes:
[0016] Obtaining a difference image between the first object image and the second object image, and obtaining a first detection result according to the difference image, wherein the first detection result is used to indicate whether the obscured target object is moved during the obscuration period;
[0017] and / or, inputting the first object image and the second object image into a preset movement prediction model to obtain a second detection result, wherein the second detection result is used to indicate whether the obscured target object is moved during the obscuration period;
[0018] And / or, determining a first moving direction of the obscured target object in a preset coordinate system based on the first object image and the second object image, determining moving directions of multiple other objects in the preset coordinate system that are present in both the first image and the second image, and obtaining a third detection result based on the first moving direction and the moving directions of the multiple other objects, wherein the third detection result is used to indicate whether the obscured target object is moved during the occlusion period.
[0019] Optionally, the determining whether the obscured target object is moved during the obscuration period according to the first object image and the second object image further includes:
[0020] When the first detection result indicates that the occluded target object has been moved during the occlusion period, the first object image and the second object image are input into a preset motion prediction model to obtain a second detection result, wherein the second detection result is used to indicate whether the occluded target object has been moved during the occlusion period.
[0021] Optionally, the determining whether the obscured target object is moved during the obscuration period according to the first object image and the second object image further includes:
[0022] When the second detection result indicates that the obscured target object has been moved during the occlusion period, a first moving direction of the obscured target object in the preset coordinate system is determined based on the first object image and the second object image, and moving directions of multiple other objects in the preset coordinate system that are present in both the first image and the second image are determined. A third detection result is obtained based on the first moving direction and the moving directions of the multiple other objects, and the third detection result is used to indicate whether the obscured target object has been moved during the occlusion period.
[0023] An object movement detection device comprises: a target object recognition unit, a first object image acquisition unit, a second object image acquisition unit and an object movement determination unit.
[0024] The target object recognition unit is used to recognize the obscured target object;
[0025] The first object image obtaining unit is used to obtain a first object image of the obscured target object in the first image, wherein the first image is an image captured before the obscured target object is not obscured, and the first object image is located in a first area in the first image;
[0026] The second object image obtaining unit is used to obtain a second object image of the obscured target object in the second image, wherein the second image is an image acquired after the obscured target object is no longer obscured, and the second object image is located in the first area of the second image;
[0027] The object movement determination unit is used to determine whether the obscured target object is moved during the obscuration period according to the first object image and the second object image.
[0028] Optionally, the target object identification unit is specifically used to determine the positional relationship between a first position mark of the target object in a captured frame of image and a second position mark of a person in the frame of image, and when the positional relationship is a preset positional relationship, determine that the target object in the frame of image is obscured by the person.
[0029] By means of the above technical scheme, the present invention provides an object movement detection method and device, which can identify an obscured target object; obtain a first object image of the obscured target object in a first image, wherein the first image is an image captured before the obscured target object is not obscured, and the first object image is located in a first area in the first image; obtain a second object image of the obscured target object in a second image, wherein the second image is an image captured after the obscured target object is no longer obscured, and the second object image is located in a first area in the second image; determine whether the obscured target object has been moved during the obscuration period according to the first object image and the second object image. The embodiment of the present invention avoids the problems of heavy system burden and waste of computer resources caused by performing movement detection on obscured target objects in an image by performing movement detection on all objects in the image.
[0030] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0032] Figure 1 A schematic diagram showing a flow chart of a method for detecting object movement provided by an embodiment of the present invention;
[0033] Figure 2 A schematic diagram of a pixel pair provided by an embodiment of the present invention is shown;
[0034] Figure 3 A schematic diagram of a difference image provided by an embodiment of the present invention is shown;
[0035] Figure 4 A schematic structural diagram of an object movement detection device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0036] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0037] like Figure 1 As shown, an object movement detection method provided by an embodiment of the present invention may include:
[0038] S100: Identify the obscured target object.
[0039] The item may be an item that the user needs to identify in the image. For example, the item may be an apple, boxed milk, hand sanitizer, cooking oil, etc. The user may pre-set one or more items to be identified. If it is determined that the item set by the user is blocked in the image, the item is determined to be the target item.
[0040] The embodiment of the present invention can identify the object in the image through a pre-trained object recognition model. The object recognition model can be a convolutional neural network model. The embodiment of the present invention can perform machine learning on the image features of the object in the image to obtain the object recognition model.
[0041] Optionally, the embodiment of the present invention can identify a target object that is blocked by a person.
[0042] Optionally, an embodiment of the present invention can determine the positional relationship between a first position mark of a target object in a captured frame of image and a second position mark of a person in the frame of image. When the positional relationship is a preset positional relationship, it is determined that the target object in the frame of image is obscured by the person.
[0043] Optionally, the first position marker and the second position marker may both be position marker points. The preset position relationship may be: the distance between the first position marker and the second position marker is less than the distance to the preset position.
[0044] Optionally, the first position marker and the second position marker are both circumscribed rectangular frames, and the preset position relationship is: the first position marker and the second position marker have an intersection.
[0045] The second position mark of the person may be a mark of a target part of the human body and / or a mark of a gaze area of the human eye.
[0046] The embodiment of the present invention can identify the human target part by using the human target part detection model obtained by training the convolutional neural network, and determine the position of the human target part mark according to the identified human target part. Optionally, the human target part can include at least one of the ear, nose, eye, neck, shoulder, elbow, wrist, waist, knee and ankle.
[0047] Optionally, an embodiment of the present invention can establish a three-dimensional model of the human head through a three-dimensional coordinate system, and combine it with existing face recognition technology to determine the orientation of the human eyes in the three-dimensional head model, and then determine the eye gaze area, thereby determining the position of the eye gaze area mark.
[0048] S200: Obtain a first object image of an occluded target object in a first image, wherein the first image is an image captured before the occluded target object is not occluded, and the first object image is located in a first area in the first image.
[0049] Optionally, the image used to identify the obscured target object and the first image are in the same image sequence, and the images in the image sequence are arranged in order of acquisition time. When the obscured target object is identified from multiple consecutive frames of images, the first image is the previous frame of the first frame of the consecutive multiple frames of images.
[0050] S300: Obtain a second object image of the obscured target object in the second image, wherein the second image is an image captured after the obscured target object is no longer obscured, and the second object image is located in a first area of the second image.
[0051] Optionally, the image used to identify the obscured target object and the second image are in the same image sequence, and the images in the image sequence are arranged in order of acquisition time. When the obscured target object is identified from multiple consecutive frames of images, the second image is the next frame of the last frame of the consecutive multiple frames of images.
[0052] The embodiment of the present invention can determine that the target object in a frame of image is no longer blocked when the positional relationship between a first position mark of the target object in a frame of image and a second position mark of a person in the frame of image is not a preset positional relationship.
[0053] S400: Determine whether the obscured target object is moved during the obstruction period according to the first object image and the second object image.
[0054] Optionally, the embodiment of the present invention may obtain a difference image between the first object image and the second object image, and obtain a first detection result according to the difference image, wherein the first detection result is used to indicate whether the obscured target object is moved during the obscuration period.
[0055] Specifically, the embodiment of the present invention can obtain multiple pixel pairs from the first object image and the second object image, each pixel pair includes a pixel located in the first object image and a pixel located in the second object image, and the positions of the pixels included in the same pixel pair in the image match. Figure 2 As shown, the first object image and the second object image are each composed of four pixels, then A1 and A2 are a pixel pair, B1 and B2 are a pixel pair, C1 and C2 are a pixel pair, and D1 and D2 are a pixel pair.
[0056] In one pixel point pair, the position of the pixel point of the first object image in the first image may be the same as the position of the pixel point of the second object image in the second image.
[0057] Optionally, the embodiment of the present invention may subtract the pixel values of two pixels in each pixel pair to obtain a difference image.
[0058] For ease of understanding, this is based on Figure 2 Based on Figure 3 For example: Assuming that the pixel value of pixel A1 is 111, the pixel value of pixel B1 is 99, the pixel value of pixel C1 is 189, the pixel value of pixel D1 is 237, the pixel value of pixel A2 is 78, the pixel value of pixel B2 is 65, the pixel value of pixel C2 is 136, and the pixel value of pixel D2 is 192, the difference image obtained is as follows: Figure 3 As shown, it is composed of pixel points with pixel values 33, 34, 53 and 45.
[0059] Optionally, an embodiment of the present invention can determine a first number of pixel points in the difference image that are greater than a preset pixel threshold, and determine whether the first number is greater than the preset number threshold; if so, it is determined that the target object has been moved during the occlusion period; if not, it is determined that the target object has not been moved during the occlusion period.
[0060] The preset pixel threshold can be set according to actual needs. For example, the preset pixel threshold can be 10.
[0061] The preset number threshold can be set according to actual needs. For example, the preset number threshold can be set to 5% of the total number of pixels in the pixel difference image.
[0062] Optionally, the embodiment of the present invention may input the first object image and the second object image into a preset movement prediction model to obtain a second detection result, where the second detection result is used to indicate whether the obscured target object is moved during the obscuration period.
[0063] Among them, the preset movement prediction model can be a deep convolutional network model designed based on the Siamese Network principle.
[0064] Optionally, in an embodiment of the present invention, when the first detection result indicates that the occluded target object has been moved during the occlusion period, the first object image and the second object image may be input into a preset motion prediction model to obtain a second detection result, wherein the second detection result is used to indicate whether the occluded target object has been moved during the occlusion period.
[0065] Specifically, when the first detection result indicates that the target object has been moved during the occlusion period, if the movement prediction result output by the preset movement prediction model is greater than the preset movement threshold, the second detection result indicates that the target object has been moved during the occlusion period, and if the movement prediction result is not greater than the preset movement threshold, the second detection result indicates that the target object has not been moved during the occlusion period. The embodiment of the present invention determines the second detection result by combining the first detection result and the movement prediction result output by the preset movement prediction model, which can solve the problem of inaccurate judgment on whether the target object has been moved due to factors such as illumination, foreign object occlusion, and noise.
[0066] Optionally, an embodiment of the present invention can determine a first moving direction of the obscured target object in a preset coordinate system based on the first object image and the second object image, determine moving directions of multiple other objects in the first image and the second image in the preset coordinate system, and obtain a third detection result based on the first moving direction and the moving directions of the multiple other objects. The third detection result is used to indicate whether the obscured target object is moved during the occlusion period.
[0067] Optionally, an embodiment of the present invention can determine a first movement direction of the occluded target object in the preset coordinate system based on the first object image and the second object image when the second detection result indicates that the occluded target object has been moved during the occlusion period, determine the movement directions of multiple other objects in the preset coordinate system that are present in both the first image and the second image, and obtain a third detection result based on the first movement direction and the movement directions of the multiple other objects. The third detection result is used to indicate whether the occluded target object has been moved during the occlusion period.
[0068] Specifically, the embodiments of the present invention can determine the number of objects with the same moving direction in the preset coordinate system, and determine whether the first moving direction of the target object is the same as the moving direction of other objects corresponding to the maximum number of objects. If they are the same, then when the maximum number of objects is greater than the preset number threshold, the third detection result indicates that the target object has not been moved during the occlusion period; and when the maximum number of objects is not greater than the preset number threshold, the third detection result indicates that the target object has been moved during the occlusion period.
[0069] The embodiment of the present invention can avoid misjudgment of object movement caused by the image acquisition device's own shaking when capturing images by determining the first moving direction of the target object and the moving directions of other objects in the same coordinate system.
[0070] It can be understood that the embodiment of the present invention can select any one of the first detection result, the second detection result and the third detection result as the final detection result of whether the target object is moved during the occlusion period.
[0071] Optionally, the embodiment of the present invention may track the target object. The embodiment of the present invention may track the target object according to the detection result of whether the blocked target object is moved during the blocking period, using a tracking method corresponding to the detection result.
[0072] Optionally, when it is determined that the obscured target object is moved during the obscuration period, the embodiment of the present invention may track the target object between different images according to image features of the target object.
[0073] Optionally, when it is determined that the obscured target object has not been moved during the obscuration period, the embodiment of the present invention may track the target object according to the position mark of the target object in different images.
[0074] Optionally, when the position mark is a circumscribed rectangle, the embodiment of the present invention can track the target object according to the intersection-and-union ratio of the position mark of the target object on one frame of image and each position mark in another frame of image. For example, the embodiment of the present invention can determine that the object corresponding to the position mark in the other frame of image with the largest intersection-and-union ratio with the position mark of the target object in the one frame of image is the target object, thereby tracking the target object.
[0075] Optionally, when the position mark is a mark point, the embodiment of the present invention can track the target object according to the image distance between the position mark of the target object on one frame image and each position mark in another frame image. For example, the embodiment of the present invention can determine that the object corresponding to the position mark in the other frame image that has the smallest distance from the position mark of the target object in the one frame image is the target object, thereby tracking the target object.
[0076] The present invention provides a method for detecting object movement, which can identify an obscured target object; obtain a first object image of the obscured target object in a first image, wherein the first image is an image captured before the obscured target object is not obscured, and the first object image is located in a first area in the first image; obtain a second object image of the obscured target object in a second image, wherein the second image is an image captured after the obscured target object is no longer obscured, and the second object image is located in a first area in the second image; determine whether the obscured target object has been moved during the obscuration period according to the first object image and the second object image. The embodiment of the present invention avoids the problem of heavy system burden and waste of computer resources caused by performing motion detection on all objects in the image by performing motion detection on the obscured target object in the image.
[0077] Optionally, the embodiment of the present invention may maintain an item library for storing item information of each item. After identifying a target item, the embodiment of the present invention may query the item information of the target item in the item library, add the item information to the item tracking list, so that the target item and the item information of the target item are stored together in the item tracking list.
[0078] Optionally, the item library can be managed manually. For example, a user can input item information of an item into the item library. Optionally, the embodiment of the present invention can determine whether the item information of the item has been stored in the item library based on the image features of the item. If it has been stored, a prompt will be given that it has been stored. If it has not been stored, the item information of the item can be newly created in the item library to avoid the existence of multiple items of the same item in the item library.
[0079] Optionally, in the embodiment of the present invention, when the first object does not exist among the objects identified from the second image, the first object may be identified as a lost object, wherein the first object is an object identified from the first image.
[0080] Optionally, an embodiment of the present invention may identify a first item as a lost item when the first item does not exist in any of the items identified from the second image and a preset number of images collected after the second image, wherein the first item is an item identified from the first image.
[0081] When an item is identified as a lost item, the item and the item information of the item are deleted from the item tracking list.
[0082] Optionally, in the embodiment of the present invention, when the second object does not exist among the objects identified from the first image, the second object may be identified as a newly added object, wherein the second object is an object identified from the second image.
[0083] Optionally, an embodiment of the present invention may identify a second object as a newly added object when the second object does not exist in any of the objects identified from the first image and a preset number of images collected before the first image, wherein the second object is an object identified from the second image.
[0084] When an item is identified as a newly added item, the item and item information of the item are added to the item tracking list.
[0085] Optionally, the embodiment of the present invention may establish a person-goods association event when the positional relationship between a first position mark of a target object in a frame of image and a second position mark of a person in the frame of image is a preset positional relationship.
[0086] Specifically, the embodiment of the present invention can calculate the overlapping time length of the first position mark and the second position mark. If the overlapping time length exceeds a preset overlapping time length threshold, a person-goods association event is established based on the human body information corresponding to the person and the object information corresponding to the target object.
[0087] If the overlap time exceeds the preset overlap time threshold and the target object is not moved during the occlusion period, the person-goods association event is determined to be an item browsing event. If the overlap time exceeds the preset overlap time threshold and the target object is moved during the occlusion period, the person-goods association event is determined to be an item taking event.
[0088] Optionally, the embodiment of the present invention may store the established person-goods association event, the user identifier corresponding to the human body, and the identifier of the item in correspondence, so as to query the person-goods association event later.
[0089] Corresponding to the above method embodiment, an embodiment of the present invention provides an object movement detection device, whose structure is as follows: Figure 4 As shown, it may include: a target item recognition unit 100 , a first item image obtaining unit 200 , a second item image obtaining unit 300 , and an item movement determination unit 400 .
[0090] The target object recognition unit 100 is used to recognize the obscured target object.
[0091] The object may be an object that the user needs to identify in the image.
[0092] The embodiment of the present invention can identify the object in the image through a pre-trained object recognition model. The object recognition model can be a convolutional neural network model. The embodiment of the present invention can perform machine learning on the image features of the object in the image to obtain the object recognition model.
[0093] Optionally, the embodiment of the present invention can identify a target object that is blocked by a person.
[0094] Optionally, the target object identification unit 100 is specifically used to determine the positional relationship between a first position mark of the target object in a captured frame of image and a second position mark of a person in the frame of image, and when the positional relationship is a preset positional relationship, determine that the target object in the frame of image is obscured by the person.
[0095] Optionally, the first position marker and the second position marker may both be position marker points. The preset position relationship may be: the distance between the first position marker and the second position marker is less than the distance to the preset position.
[0096] Optionally, the first position marker and the second position marker are both circumscribed rectangular frames, and the preset position relationship is: the first position marker and the second position marker have an intersection.
[0097] The second position mark of the person may be a mark of a target part of the human body and / or a mark of a gaze area of the human eye.
[0098] The embodiment of the present invention can identify the human target part by using the human target part detection model obtained by training the convolutional neural network, and determine the position of the human target part mark according to the identified human target part. Optionally, the human target part can include at least one of the ear, nose, eye, neck, shoulder, elbow, wrist, waist, knee and ankle.
[0099] Optionally, an embodiment of the present invention can establish a three-dimensional model of the human head through a three-dimensional coordinate system, and combine it with existing face recognition technology to determine the orientation of the human eyes in the three-dimensional head model, and then determine the eye gaze area, thereby determining the position of the eye gaze area mark.
[0100] The first object image obtaining unit 200 is used to obtain a first object image of the occluded target object in the first image, wherein the first image is an image captured before the occluded target object is not occluded, and the first object image is located in a first area in the first image.
[0101] Optionally, the image used to identify the occluded target object and the first image are located in the same image sequence, and the images in the image sequence are arranged in chronological order according to the acquisition time; when the occluded target object is identified from multiple consecutive frames of images, the first image is the previous frame image of the first frame image in the multiple consecutive frames of images.
[0102] The second object image obtaining unit 300 is used to obtain a second object image of the occluded target object in the second image, wherein the second image is an image captured after the occluded target object is no longer occluded, and the second object image is located in a first area in the second image.
[0103] Optionally, the image used to identify the occluded target object and the second image are located in the same image sequence, and the images in the image sequence are arranged in chronological order according to the acquisition time; when the occluded target object is identified from multiple consecutive frames of images, the second image is the frame image after the last frame image in the multiple consecutive frames of images.
[0104] The embodiment of the present invention can determine that the target object in a frame of image is no longer blocked when the positional relationship between a first position mark of the target object in a frame of image and a second position mark of a person in the frame of image is not a preset positional relationship.
[0105] The object movement determination unit 400 is used to determine whether the blocked target object is moved during the blocking period according to the first object image and the second object image.
[0106] Optionally, the object movement determination unit 400 may be specifically configured to obtain a difference image between the first object image and the second object image, and obtain a first detection result based on the difference image, wherein the first detection result is used to indicate whether the occluded target object is moved during the occlusion period.
[0107] Specifically, an embodiment of the present invention can obtain multiple pixel pairs from the first object image and the second object image, each pixel pair includes a pixel located in the first object image and a pixel located in the second object image, and the positions of the pixels included in the same pixel pair in the image match.
[0108] In one pixel point pair, the position of the pixel point of the first object image in the first image may be the same as the position of the pixel point of the second object image in the second image.
[0109] Optionally, the embodiment of the present invention may subtract the pixel values of two pixels in each pixel pair to obtain a difference image.
[0110] Optionally, an embodiment of the present invention can determine a first number of pixel points in the difference image that are greater than a preset pixel threshold, and determine whether the first number is greater than the preset number threshold; if so, it is determined that the target object has been moved during the occlusion period; if not, it is determined that the target object has not been moved during the occlusion period.
[0111] The preset pixel threshold can be set according to actual needs.
[0112] Among them, the preset quantity threshold can be set according to actual needs.
[0113] Optionally, the object movement determination unit 400 can be specifically used to input the first object image and the second object image into a preset movement prediction model to obtain a second detection result, and the second detection result is used to indicate whether the occluded target object is moved during the occlusion period.
[0114] Among them, the preset movement prediction model can be a deep convolutional network model designed based on the Siamese Network principle.
[0115] Optionally, the object movement determination unit 400 can be specifically used to input the first object image and the second object image into a preset movement prediction model to obtain a second detection result when the first detection result indicates that the occluded target object has been moved during the occlusion period, and the second detection result is used to indicate whether the occluded target object has been moved during the occlusion period.
[0116] Specifically, when the first detection result indicates that the target object has been moved during the occlusion period, if the movement prediction result output by the preset movement prediction model is greater than the preset movement threshold, the second detection result indicates that the target object has been moved during the occlusion period, and if the movement prediction result is not greater than the preset movement threshold, the second detection result indicates that the target object has not been moved during the occlusion period. The embodiment of the present invention determines the second detection result by combining the first detection result and the movement prediction result output by the preset movement prediction model, which can solve the problem of inaccurate judgment on whether the target object has been moved due to factors such as illumination, foreign object occlusion, and noise.
[0117] Optionally, the object movement determination unit 400 can be specifically used to determine a first movement direction of the obscured target object in a preset coordinate system based on the first object image and the second object image, determine the movement directions of multiple other objects existing in the first image and the second image in the preset coordinate system, and obtain a third detection result based on the first movement direction and the movement directions of the multiple other objects. The third detection result is used to indicate whether the obscured target object is moved during the occlusion period.
[0118] Optionally, the object movement determination unit 400 can be specifically used to determine a first movement direction of the occluded target object in a preset coordinate system based on the first object image and the second object image when the second detection result indicates that the occluded target object has been moved during the occlusion period, determine the movement directions of multiple other objects existing in the first image and the second image in the preset coordinate system, and obtain a third detection result based on the first movement direction and the movement directions of the multiple other objects. The third detection result is used to indicate whether the occluded target object has been moved during the occlusion period.
[0119] Specifically, the embodiments of the present invention can determine the number of objects with the same moving direction in the preset coordinate system, and determine whether the first moving direction of the target object is the same as the moving direction of other objects corresponding to the maximum number of objects. If they are the same, then when the maximum number of objects is greater than the preset number threshold, the third detection result indicates that the target object has not been moved during the occlusion period; and when the maximum number of objects is not greater than the preset number threshold, the third detection result indicates that the target object has been moved during the occlusion period.
[0120] The embodiment of the present invention can avoid misjudgment of object movement caused by the image acquisition device's own shaking when capturing images by determining the first moving direction of the target object and the moving directions of other objects in the same coordinate system.
[0121] It can be understood that the embodiment of the present invention can select any one of the first detection result, the second detection result and the third detection result as the final detection result of whether the target object is moved during the occlusion period.
[0122] Optionally, another object movement detection device provided by an embodiment of the present invention further includes a target object tracking unit.
[0123] The target object tracking unit is used to track the target object. The target object tracking unit can track the target object using a tracking method corresponding to the detection result of whether the blocked target object is moved during the blocking period determined by the object movement determination unit 400.
[0124] Optionally, the target object tracking unit is used to track the target object between different images according to image features of the target object when the object movement determination unit 400 determines that the blocked target object has been moved during the blocking period.
[0125] Optionally, the target object tracking unit is used to track the target object according to the position mark of the target object in different images when the object movement determination unit 400 determines that the blocked target object has not been moved during the blocking period.
[0126] Optionally, when the position mark is a circumscribed rectangle, the target object tracking unit may track the target object according to an intersection-and-union ratio of the position mark of the target object on one frame of image and each position mark in another frame of image.
[0127] Optionally, when the position mark is a mark point, the target object tracking unit may track the target object according to the image distance between the position mark of the target object on one frame of image and each position mark in another frame of image.
[0128] The present invention provides an object movement detection device, which can identify an obscured target object; obtain a first object image of the obscured target object in a first image, wherein the first image is an image captured before the obscured target object is not obscured, and the first object image is located in a first area in the first image; obtain a second object image of the obscured target object in a second image, wherein the second image is an image captured after the obscured target object is no longer obscured, and the second object image is located in a first area in the second image; determine whether the obscured target object has been moved during the obscuration period based on the first object image and the second object image. The embodiment of the present invention avoids the problems of heavy system burden and waste of computer resources caused by performing movement detection on all objects in the image by performing movement detection on the obscured target object in the image.
[0129] Optionally, another object movement detection device provided by an embodiment of the present invention may further include: a lost object identification unit.
[0130] Optionally, the lost item identification unit may be configured to identify the first item as a lost item when the first item does not exist among the items identified from the second image, wherein the first item is an item identified from the first image.
[0131] Optionally, the lost item identification unit may be used to identify the first item as a lost item when the first item does not exist among all items identified from the second image and a preset number of images collected after the second image, wherein the first item is an item identified from the first image.
[0132] Optionally, another object movement detection device provided by an embodiment of the present invention may further include: a new object identification unit.
[0133] Optionally, the newly added object identification unit may be configured to identify the second object as a newly added object when the second object does not exist among the objects identified from the first image, wherein the second object is an object identified from the second image.
[0134] Optionally, the newly added object identification unit may be used to identify the second object as a newly added object when the second object does not exist among the objects identified from the first image and a preset number of images collected before the first image, wherein the second object is an object identified from the second image.
[0135] Optionally, another device for detecting movement of an object provided by an embodiment of the present invention may further include: a person-goods association event establishing unit.
[0136] The person-cargo association event establishing unit is used to establish a person-cargo association event when the position relationship between the first position mark of the target object in a frame of image and the second position mark of the person in the frame of image is a preset position relationship.
[0137] Specifically, the embodiment of the present invention can calculate the overlapping time length of the first position mark and the second position mark. If the overlapping time length exceeds a preset overlapping time length threshold, a person-goods association event is established based on the human body information corresponding to the person and the object information corresponding to the target object.
[0138] If the overlap time exceeds the preset overlap time threshold and the target object is not moved during the occlusion period, the person-goods association event is determined to be an item browsing event. If the overlap time exceeds the preset overlap time threshold and the target object is moved during the occlusion period, the person-goods association event is determined to be an item taking event.
[0139] Optionally, the embodiment of the present invention may store the established person-goods association event, the user identifier corresponding to the human body, and the identifier of the item in correspondence, so as to query the person-goods association event later.
[0140] The object movement detection device includes a processor and a memory. The target object recognition unit 100, the first object image acquisition unit 200, the second object image acquisition unit 300, and the object movement determination unit 400 are all stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions.
[0141] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to detect the motion of the target objects blocked in the image, avoiding the problem of heavy system burden and waste of computer resources caused by performing motion detection on all objects in the image.
[0142] An embodiment of the present invention provides a storage medium on which a program is stored. When the program is executed by a processor, the object movement detection method is implemented.
[0143] An embodiment of the present invention provides a processor, which is used to run a program, wherein the object movement detection method is executed when the program is running.
[0144] An embodiment of the present invention provides an electronic device, which includes at least one processor, at least one memory connected to the processor, and a bus; wherein the processor and the memory communicate with each other through the bus; the processor is used to call program instructions in the memory to execute the above-mentioned object movement detection method. The electronic device in this article can be a server, a PC, a PAD, a mobile phone, etc.
[0145] The present application also provides a computer program product, which, when executed on an electronic device, is suitable for executing a program initialized with the steps of the above-mentioned object movement detection method.
[0146] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, electronic devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processor of the computer or other programmable device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0147] In a typical configuration, an electronic device includes one or more processors (CPU), a memory, and a bus. The electronic device may also include an input / output interface, a network interface, and the like.
[0148] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.
[0149] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0150] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0151] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0152] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.
Claims
1. A method for detecting object movement, It is characterized in that include: Identify occluded objects; Obtaining a first object image of the obscured target object in a first image, wherein the first image is an image captured before the obscured target object is not obscured, and the first object image is located in a first area of the first image; Obtaining a second object image of the obscured target object in the second image, wherein the second image is an image captured after the obscured target object is no longer obscured, and the second object image is located in the first area of the second image; determining whether the obscured target object is moved during the obscuration period according to the first object image and the second object image; Wherein, determining whether the blocked target object is moved during the blocking period according to the first object image and the second object image includes: Obtaining a difference image between the first object image and the second object image, and obtaining a first detection result according to the difference image, wherein the first detection result is used to indicate whether the obscured target object is moved during the obscuration period; When the first detection result indicates that the obscured target object has been moved during the obscuration period, inputting the first object image and the second object image into a preset movement prediction model to obtain a second detection result, wherein the second detection result is used to indicate whether the obscured target object has been moved during the obscuration period; When the second detection result indicates that the obscured target object has been moved during the occlusion period, a first moving direction of the obscured target object in a preset coordinate system is determined based on the first object image and the second object image, and moving directions of multiple other objects in the preset coordinate system are determined, and a third detection result is obtained based on the first moving direction and the moving directions of the multiple other objects. The third detection result is used to indicate whether the obscured target object has been moved during the occlusion period.
2. The method according to claim 1, It is characterized in that The identifying the obscured target object comprises: Determine a positional relationship between a first position mark of the target object in a captured frame of image and a second position mark of a person in the frame of image, and when the positional relationship is a preset positional relationship, determine that the target object in the frame of image is blocked by the person.
3. The method according to claim 2, It is characterized in that The first position mark and the second position mark are both circumscribed rectangular frames, and the preset position relationship is: the first position mark and the second position mark have an intersection.
4. The method according to claim 1, It is characterized in that The image used to identify the obscured target object and the first image are in the same image sequence, and the images in the image sequence are arranged in sequence according to the chronological order of acquisition time; when the obscured target object is identified from multiple consecutive frames of images, the first image is the previous frame image of the first frame image in the multiple consecutive frames of images.
5. The method according to claim 1, It is characterized in that The image used to identify the obscured target object and the second image are located in the same image sequence, and the images in the image sequence are arranged in sequence according to the chronological order of acquisition time; when the obscured target object is identified from multiple consecutive frames of images, the second image is the frame image after the last frame image in the multiple consecutive frames of images.
6. An object movement detection device, It is characterized in that include: a target object recognition unit, a first object image acquisition unit, a second object image acquisition unit, and an object movement determination unit, The target object recognition unit is used to recognize the obscured target object; The first object image obtaining unit is used to obtain a first object image of the obscured target object in the first image, wherein the first image is an image captured before the obscured target object is not obscured, and the first object image is located in a first area in the first image; The second object image obtaining unit is used to obtain a second object image of the obscured target object in the second image, wherein the second image is an image acquired after the obscured target object is no longer obscured, and the second object image is located in the first area of the second image; The object movement determination unit is used to determine whether the blocked target object is moved during the blocking period according to the first object image and the second object image; The object movement determination unit determines whether the blocked target object is moved during the blocking period according to the first object image and the second object image, including: Obtaining a difference image between the first object image and the second object image, and obtaining a first detection result according to the difference image, wherein the first detection result is used to indicate whether the obscured target object is moved during the obscuration period; When the first detection result indicates that the obscured target object has been moved during the obscuration period, inputting the first object image and the second object image into a preset movement prediction model to obtain a second detection result, wherein the second detection result is used to indicate whether the obscured target object has been moved during the obscuration period; When the second detection result indicates that the obscured target object has been moved during the occlusion period, a first moving direction of the obscured target object in a preset coordinate system is determined based on the first object image and the second object image, and moving directions of multiple other objects in the preset coordinate system are determined, and a third detection result is obtained based on the first moving direction and the moving directions of the multiple other objects. The third detection result is used to indicate whether the obscured target object has been moved during the occlusion period.
7. The device according to claim 6, It is characterized in that The target object identification unit is specifically used to determine the positional relationship between a first position mark of the target object in a captured frame of image and a second position mark of a person in the frame of image, and when the positional relationship is a preset positional relationship, determine that the target object in the frame of image is obscured by the person.
