Monitoring method and device, monitoring equipment and storage medium
By acquiring a target image set and predicting the motion trajectory in the monitoring device, the problem of low video recording retention rate in the monitoring device is solved, achieving a more efficient monitoring effect and user experience.
Patent Information
- Application Number
- CN202410332776.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-21
- Publication Date
- 2025-09-23
AI Technical Summary
In existing monitoring equipment, due to the time difference between the startup time of the image acquisition module and the AI detection module, there is a deviation between the detected position and the actual position of the monitored object, and the video recording retention rate is low.
After the image acquisition module is started, the target image set is obtained through the AI detection module, the image is split and processed based on the timestamp, the motion trajectory of the monitored object is predicted, and the camera position is adjusted to ensure the integrity of the recording.
It improves the success rate of recording, ensures that the monitored object does not leave the monitoring range, and improves the monitoring effect and user experience.
Smart Images
Figure CN120689805A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of monitoring equipment, and in particular to a monitoring method, apparatus, monitoring equipment, and storage medium. Background Art
[0002] For some monitoring devices, in order to balance the impact on power consumption, the main control is generally in the off mode. However, when a monitored object appears, it is necessary to be able to capture and save the corresponding video as quickly and completely as possible.
[0003] Some surveillance devices currently on the market can combine camera quick startup and AI detection to quickly capture images of the monitored area and save the corresponding images. To a certain extent, they can completely capture and save videos.
[0004] However, when the main control chip is started, there is a time difference between the moment when the image acquisition module is started and the moment when the AI detection module is started, which causes a deviation between the detection position of the monitored object detected by the AI detection module and the actual position of the monitored object. In addition, due to the time difference between the above-mentioned startup completion moments, when the monitored object does not exist in the first few frames of images collected by the image acquisition module, the AI detection module cannot detect the monitored object from the first few frames of images, which can easily lead to the inability to save the video, that is, the video saving rate in the related technology is low. For example, in the related technology, when the monitored object is a person, the schematic diagram of the human coordinate trajectory in the monitoring video can be referred to Figure 5 shown. Summary of the Invention
[0005] In view of this, in order to solve the above-mentioned technical problems of deviation between the detected position and the actual position of the monitored object in the monitoring equipment and low video preservation rate, the present disclosure provides a monitoring method, apparatus, monitoring equipment and storage medium.
[0006] According to a first aspect of an embodiment of the present disclosure, a monitoring method is provided, the monitoring method comprising:
[0007] After the image acquisition module is started, the image data of the monitored object is collected and stored by the image acquisition module;
[0008] After the AI detection module is started, based on the startup completion time of the AI detection module and the timestamp of the first frame of the stored image data, a target image set is obtained, where the target image set includes N target images selected from the image data; where N is a positive integer greater than or equal to 3;
[0009] The target image set is processed by the AI detection module to obtain the motion trajectory of the monitored object between the acquisition time of the Nth target image and the current time.
[0010] In an optional embodiment, acquiring the target image set based on the startup completion time of the AI detection module and the timestamp of the first frame image in the stored image data includes:
[0011] Obtaining a startup time difference between the image acquisition module and the AI detection module through the startup completion moment and the timestamp of the first frame image;
[0012] Split the startup time difference into N-1 time intervals;
[0013] Based on N-1 time intervals, N images are selected from the stored image data as the N target images.
[0014] In an optional implementation manner, splitting the startup time difference into N-1 time intervals includes:
[0015] The startup time difference is evenly divided to obtain N-1 equal time intervals.
[0016] In an optional embodiment, the processing of the target image set by the AI detection module to obtain the motion trajectory of the monitored object between the capture time of the Nth target image and the current time includes:
[0017] Processing the target image through the AI detection module to obtain target position information corresponding to the target image; wherein the target position information is the position information of the monitored object at the time of acquisition of the target image;
[0018] Obtaining the speed information and acceleration information of the monitored object corresponding to the Nth target image through the N target position information corresponding to the target image set;
[0019] The motion trajectory is obtained through the velocity information and the acceleration information.
[0020] In an optional embodiment, the speed information includes a first component speed in a first direction and a second component speed in a second direction, the first direction and the second direction being perpendicular to each other;
[0021] The obtaining, through the N target position information corresponding to the target image set, the speed information and acceleration information of the monitored object corresponding to the Nth target image, includes:
[0022] Obtaining a first distance in the first direction and a second distance in the second direction using the Mth target position information and the Nth target position information; wherein M is a minimum positive integer greater than or equal to N / 2;
[0023] Obtaining the first component velocity by using the first distance, a timestamp of the Mth target image, and a timestamp of the Nth target image;
[0024] The second component speed is obtained by using the second distance, the timestamp of the Mth target image, and the timestamp of the Nth target image.
[0025] In an optional embodiment, the acceleration information includes a first component acceleration in a first direction and a second component acceleration in a second direction, wherein the first direction and the second direction are perpendicular to each other;
[0026] The obtaining of the speed information and acceleration information of the monitored object corresponding to the Nth target image by the N target position information corresponding to the target image set includes:
[0027] Obtaining a first distance in the first direction and a second distance in the second direction using the Mth target position information and the Nth target position information; wherein M is a minimum positive integer greater than or equal to N / 2;
[0028] Obtaining a third distance in the first direction and a fourth distance in the second direction through the first target position information and the M-th target position information;
[0029] Obtaining the first component acceleration through the first distance, the third distance, a timestamp of the first target image, a timestamp of the Mth target image, and a timestamp of the Nth target image;
[0030] The second component acceleration is obtained by using the second distance, the fourth distance, the timestamp of the first target image, the timestamp of the Mth target image, and the timestamp of the Nth target image.
[0031] In an optional embodiment, after obtaining the motion trajectory of the monitored object between the capture time of the Nth target image and the current time, the monitoring method includes:
[0032] Based on the motion trajectory, the position of a surveillance camera is adjusted; wherein the surveillance camera is used to monitor the surveillance object.
[0033] According to a second aspect of an embodiment of the present disclosure, a monitoring device is provided, the monitoring device comprising:
[0034] An image acquisition module, configured to acquire and store image data of a monitored object after the image acquisition module is started;
[0035] an AI detection module, configured to, after the AI detection module is started, acquire a target image set based on a startup completion time of the AI detection module and a timestamp of a first frame of the stored image data, the target image set comprising N target images selected from the image data; wherein N is a positive integer greater than or equal to 3;
[0036] The AI detection module is further used to process the target image set to obtain the motion trajectory of the monitored object between the acquisition time of the Nth target image and the current time.
[0037] According to a third aspect of an embodiment of the present disclosure, a monitoring device is provided, the monitoring device comprising:
[0038] processor;
[0039] a memory for storing instructions executable by the processor;
[0040] Wherein, the processor is configured to execute the monitoring method as described in any one of the first aspects.
[0041] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided, which stores one or at least one program, and the one or at least one program can be executed by one or at least one processor to implement the monitoring method described in any one of the first aspects.
[0042] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: In the present disclosure, N images are extracted as target images from the image data collected during the time period between the timestamp of the image acquisition module collecting the first frame of image (that is, the moment when the image acquisition module is started and completed) and the moment when the AI detection module is started and completed, which can better ensure that the AI detection module detects the monitored object from the N target images, so as to better improve the success rate of recording. In addition, based on the above N images, the motion trajectory between the acquisition moment of the above N-th image (that is, the last frame of image) and the current moment is predicted, which can play a greater role in scenarios where the motion trajectory needs to be predicted. For example, the motion path of the monitored object can be drawn, and the camera can also be controlled to rotate to the corresponding acquisition position in advance according to the predicted motion trajectory, so as to better avoid the monitored object from walking out of the monitoring screen and being unable to track the monitored object. This method can better ensure the monitoring effect, provide better security protection for users, and improve the user experience.
[0043] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0045] Figure 1 The figure is a flowchart of a monitoring method according to an exemplary embodiment.
[0046] Figure 2 is a flowchart of a monitoring method according to another exemplary embodiment.
[0047] Figure 3 is a block diagram of a monitoring device according to an exemplary embodiment.
[0048] Figure 4 is a block diagram of a monitoring device according to an exemplary embodiment.
[0049] Figure 5 It is a schematic diagram of the trajectory of human coordinates in surveillance video according to relevant technology.
[0050] Figure 6 The figure is a schematic diagram showing the trajectory of coordinates of a person in a surveillance video according to an exemplary embodiment. DETAILED DESCRIPTION
[0051] The following will describe the embodiments of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand the other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for the purpose of illustrating the present application and are not intended to limit the scope of protection of the present application.
[0052] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. Therefore, the illustrations only show components related to the present application rather than the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.
[0053] The following will describe the embodiments of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand the other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for the purpose of illustrating the present application and are not intended to limit the scope of protection of the present application.
[0054] The embodiments of the present disclosure provide a monitoring method, apparatus, monitoring equipment and storage medium. In the present disclosure, N images are extracted as target images from the image data collected during the time period between the timestamp of the first frame image collected by the image acquisition module (that is, the moment when the image acquisition module is started and completed) and the moment when the AI detection module is started and completed. This can better ensure that the AI detection module detects the monitored object from the N target images, so as to better improve the success rate of video recording. In addition, based on the above-mentioned N images, the motion trajectory between the acquisition moment of the above-mentioned N-th image (that is, the last frame image) and the current moment is predicted, which can play a greater role in scenarios where the motion trajectory needs to be predicted. For example, the motion path of the monitored object can be drawn, and the camera can also be controlled to rotate to the corresponding acquisition position in advance according to the predicted motion trajectory, so as to better avoid the monitored object from walking out of the monitoring screen and being unable to track the monitored object. This method can better ensure the monitoring effect, provide better security protection for users, and improve the user experience.
[0055] In an exemplary embodiment, a monitoring method is provided, which can be applied to a monitoring device. Figure 1 As shown, the method may include:
[0056] S110, after the image acquisition module is started, the image acquisition module acquires and stores image data of the monitored object;
[0057] S120. After the AI detection module is started, based on the startup completion time of the AI detection module and the timestamp of the first frame of the stored image data, obtain a target image set, where the target image set includes N target images selected from the image data; where N is a positive integer greater than or equal to 3;
[0058] S130: Process the target image set through the AI detection module to obtain the motion trajectory of the monitored object between the capture time of the Nth target image and the current time.
[0059] In step S130, the monitoring device may include a microcontroller and a main control chip. After receiving the trigger signal, the microcontroller may send a wake-up signal to the main control chip to wake it up. The trigger signal is generated when the monitoring device detects the presence of a monitored object within the monitoring area. The monitored object can be any object that can be monitored by a camera, such as a person, animal, vehicle, moving box, or other movable object. The trigger signal can switch the main control chip from an off state to an awakened state. When the main control chip is in the awakened state, it can control the activation of various monitoring modules to ensure subsequent monitoring of the monitored object.
[0060] After the main control chip receives the wake-up signal, it can start only the necessary modules based on the signal reason of the wake-up signal, and temporarily deactivate unnecessary modules to start the image acquisition module and AI detection module as quickly as possible. For example, when the wake-up signal is a Wi-Fi wake-up, the network communication module and database module can be initialized first to ensure that the user can see the real-time monitoring screen and historical images as soon as possible when waking the device via Wi-Fi. When the wake-up signal is a mobile wake-up, the current monitoring screen may need to be saved, so the image acquisition module and AI detection module can be initialized first, and the remaining modules can be initialized later.
[0061] The image acquisition module and the AI detection module generally require different startup times. Generally, the image acquisition module is started first, followed by the AI detection module. Once the image acquisition module is started, image data can be acquired. This means that the image acquisition module can capture image data of the monitored object. This means that before the AI detection module is started, the image acquisition module has already captured a portion of the monitored object's image data.
[0062] Among them, in order to facilitate the AI detection module to use the above-mentioned image data, a cache area can be configured in the monitoring device. After the image acquisition module collects the image data of the monitored object, the above-mentioned image data can be stored in the cache area, so that the AI detection module can subsequently obtain the above-mentioned image data from the cache area.
[0063] In step S120, after the AI detection module is activated, i.e., after the AI detection module completes startup, the startup completion time of the AI detection module can be recorded. Then, the timestamp of the first frame of the image data in the buffer can be obtained. Based on the time corresponding to the timestamp and the startup completion time, the AI detection module can select N images from the image data in the buffer as N target images in a predetermined manner. These N target images can constitute the target image set. In this way, the AI detection module can obtain the target image set.
[0064] Among them, when N images are selected as target images from the image data in the cache in a set manner, the time difference between the two can be determined based on the startup completion time of the AI detection module and the timestamp of the first frame image in the cache, and used as the startup time difference between the image acquisition module and the AI detection module, that is, the time difference between the moment when the image acquisition module completes the startup and the moment when the AI detection module completes the startup.
[0065] The startup time difference is then divided into N-1 time intervals. Based on the N-1 time intervals, N images are selected from the cache as N target images. For example, when selecting the target image, the first frame image in the cache can be used as the first target image, and then the second target image is selected at the first time interval. That is, the second target image is obtained by the timestamp of the first target image and the first time interval. The sum of the timestamp of the first target image and the first time interval is recorded as the target moment, and the image corresponding to the timestamp closest to this target moment is selected as the second target image. By analogy, N target images can be obtained.
[0066] When splitting the startup time difference, the startup time difference can be evenly divided into N-1 parts, resulting in N-1 equal time intervals. Then, based on the time intervals, N images are selected from the cache as target images. When selecting a target image, the first frame in the cache can be used as the first target image, and then other target images are selected based on time intervals. Selecting target images by evenly dividing the obtained time intervals can improve the efficiency of splitting the startup time difference and the efficiency of selecting target images.
[0067] In some embodiments,
[0068] The overall architecture of the monitoring device consists of a microcontroller and a main control chip. Upon receiving a trigger signal, the microcontroller wakes up the main control chip, which then activates the image acquisition module and the AI detection module. After the image acquisition module is activated, the captured image data of the monitored object is stored in a buffer until the AI detection module is initialized and activated.
[0069] In this embodiment, N can be a positive integer greater than or equal to 3, for example, N is 3. When N is 3, after the AI detection module is ready (i.e., after the AI detection module is started), three frames of images can be selected from the cache at a time interval of (startup completion time - timestamp of the first frame of image) / 2 based on the startup completion time of the AI detection module and the timestamp of the first frame of image captured by the image acquisition module. The three selected frames of images are used as target images. Subsequently, the monitoring device can perform trajectory prediction of the monitored object based on the AI detection module and the three target images to ensure the reliability of monitoring.
[0070] It should be noted that, in addition to obtaining the target image from the collected image data in the above-mentioned manner, the target image may also be obtained in other manners, which are not limited thereto.
[0071] In step S130, after obtaining a target image set consisting of N target images, image conversion can be performed on the target images to convert the target images into target data required by the AI detection module, so that the AI detection module can process the above-mentioned converted target data.
[0072] It should be noted that the converted target data can be an RGB image or an RGBA image, or data in other formats that can be processed by the AI detection module, without limitation.
[0073] Among them, the AI detection module can process the above-mentioned target data to obtain the motion data of the monitored object (such as speed information and / or acceleration information, etc.), and then predict the motion trajectory within the target time period based on the motion data of the monitored object. The target time period may include the time period between the acquisition time of the Nth target image and the current time. The acquisition time of the Nth target image can be determined based on its timestamp. The current time is the moment when the AI detection module determines the motion data of the monitored object based on the target data, that is, the starting moment when the monitoring device predicts the motion trajectory based on the above-mentioned motion data.
[0074] Among them, after determining the motion trajectory, the next location of the monitored object can be predicted, and then the position of the monitoring camera of the monitoring equipment (the monitoring camera is used to monitor the above-mentioned monitored object) can be adjusted so that the monitoring area of the monitoring camera is aligned with the above-mentioned position to ensure that the monitoring camera can capture the image of the monitored object, avoid the monitored object from walking out of the monitoring range of the monitoring camera, and better realize the monitoring of the monitored object.
[0075] In this method, N images are extracted as target images from the image data collected during the time period between the timestamp of the first frame of image collected by the image acquisition module (that is, the moment when the image acquisition module is started and completed) and the moment when the AI detection module is started and completed. This can better ensure that the AI detection module detects the monitored object from the N target images, so as to better improve the success rate of recording. In addition, based on the above N images, the motion trajectory between the acquisition moment of the above N-th image (that is, the last frame of image) and the current moment is predicted. This can play a greater role in scenarios where it is necessary to predict the motion trajectory. For example, the motion path of the monitored object can be drawn, and the camera can be controlled to rotate to the corresponding acquisition position in advance according to the predicted motion trajectory, so as to better avoid the monitored object from walking out of the monitoring screen and being unable to track the monitored object. This method can better ensure the monitoring effect, provide better security protection for users, and improve the user experience.
[0076] In an exemplary embodiment, a monitoring method is provided, which is applied to a monitoring device. Figure 2 As shown, in this method, the target image set is processed by the AI detection module to obtain the motion trajectory of the monitored object between the acquisition time of the Nth target image and the current time, which may include:
[0077] S210, processing the target image through the AI detection module to obtain target position information corresponding to the target image;
[0078] S220, obtaining speed information and acceleration information of the monitored object corresponding to the Nth target image through the N target position information corresponding to the target image set;
[0079] S230 : Obtain a motion trajectory through the velocity information and acceleration information.
[0080] In step S210 , the target position information is the position information of the monitored object at the time of capturing the target image.
[0081] For example, for the Nth target image, this target image is converted to obtain the target data corresponding to this target image, and then the above target data is processed based on the AI detection module to obtain the position information of the monitored object at the time of acquisition of the above target image. This position information is the target position information corresponding to the above Nth target image.
[0082] In step S220, after obtaining the target position information corresponding to each target image, the motion data of the monitored object can be calculated based on the N target position information. For example, the target position information can be represented by coordinates. This may include coordinates in a first direction (X) and a second direction (Y), where the first and second directions are perpendicular to each other.
[0083] The motion data may include speed information and acceleration information. The speed information may include a first speed component in a first direction and a second speed component in a second direction. When determining the speed information, the first distance in the first direction and the second distance in the second direction may be obtained using the Mth target position information and the Nth target position information; where M is the smallest positive integer greater than or equal to N / 2. The first speed component may then be obtained using the first distance and the timestamp of the Mth target image and the timestamp of the Nth target image. The second speed component may also be obtained using the second distance and the timestamp of the Mth target image and the timestamp of the Nth target image.
[0084] In some embodiments,
[0085] The first direction can be denoted as the X direction, and the second direction can be denoted as the Y direction. The component velocity in the X direction is denoted as Vx, and the component velocity in the Y direction is denoted as Vy. The X-direction position information of the H-th target position information among the N target position information is denoted as centerHx, and the Y-direction position information of the H-th target position information is denoted as centerHy. The acquisition time of the target image corresponding to the H-th target position information can be denoted as tH, where H is a positive integer greater than or equal to 1 and less than or equal to N.
[0086] Where Vx = (centerNx - centerMx) / (tN - tM);
[0087] Vy=(centerNy-centerMy) / (tN-tM);
[0088] M is the smallest positive integer greater than or equal to N / 2.
[0089] For example, when N is 3, Vx = (center3x - center2x) / (t3 - t2), Vy = (center3y - center2y) / (t3 - t2).
[0090] For another example, when N is 4, Vx = (center4x - center2x) / (t4 - t2), Vy = (center4y - center2y) / (t4 - t2).
[0091] In addition, the speed information and position information can also be divided into three directions, for example, the first direction (X), the second direction (Y) and the third direction (Z), and then the corresponding component speeds are calculated based on the position information in each direction. The calculation method of the component speed in a single direction can refer to the above-mentioned implementation methods and will not be repeated here.
[0092] It should be noted that, in addition to determining the speed information in the above manner, the speed information may also be determined in other manners, which are not limited thereto.
[0093] In some embodiments,
[0094] The first direction can be denoted as the X direction, and the second direction can be denoted as the Y direction. The component velocity in the X direction is denoted as Vx, and the component velocity in the Y direction is denoted as Vy. The X-direction position information of the H-th target position information among the N target position information is denoted as centerHx, and the Y-direction position information of the H-th target position information is denoted as centerHy. The acquisition time of the target image corresponding to the H-th target position information can be denoted as tH, where H is a positive integer greater than or equal to 1 and less than or equal to N.
[0095] Where Vx = (centerNx - center1x) / (tN - t1);
[0096] Vy=(centerNy-center1y) / (tN-t1).
[0097] For example, when N is 3, Vx=(center3x-center1x) / (t3-t1), Vy=(center3y-center1y) / (t3-t1).
[0098] For another example, when N is 4, Vx = (center4x - center1x) / (t4 - t1), Vy = (center4y - center1y) / (t4 - t1).
[0099] In some embodiments,
[0100] The first direction can be denoted as the X direction, and the second direction can be denoted as the Y direction. The component velocity in the X direction is denoted as Vx, and the component velocity in the Y direction is denoted as Vy. The X-direction position information of the H-th target position information among the N target position information is denoted as centerHx, and the Y-direction position information of the H-th target position information is denoted as centerHy. The acquisition time of the target image corresponding to the H-th target position information can be denoted as tH, where H is a positive integer greater than or equal to 1 and less than or equal to N.
[0101] Where Vx = (centerNx - centerEx) / (tN - tE);
[0102] Vy=(centerNy-centerEy) / (tN-tE);
[0103] E=N-1.
[0104] For example, when N is 3, Vx = (center3x - center2x) / (t3 - t2), Vy = (center3y - center2y) / (t3 - t2).
[0105] For another example, when N is 4, Vx = (center4x - center3x) / (t4 - t3), Vy = (center4y - center3y) / (t4 - t3).
[0106] In this step, the acceleration information may include a first component acceleration in the first direction and a second component acceleration in the second direction. When determining the acceleration information, the first distance in the first direction and the second distance in the second direction can be obtained using the Mth target position information and the Nth target position information; where M can be a minimum positive integer greater than or equal to N / 2. Then, the third distance in the first direction and the fourth distance in the second direction can be obtained using the first target position information and the Mth target position information. The first component acceleration can also be obtained using the first distance, the third distance, the timestamp of the first target image, the timestamp of the Mth target image, and the timestamp of the Nth target image. Furthermore, the second component acceleration can be obtained using the second distance, the fourth distance, the timestamp of the first target image, the timestamp of the Mth target image, and the timestamp of the Nth target image.
[0107] In some embodiments,
[0108] The first direction can be denoted as the X direction, and the second direction can be denoted as the Y direction. The component acceleration in the X direction is denoted as DVx, and the component velocity in the Y direction is denoted as DVy. The X-direction position information of the H-th target position information among the N target position information is denoted as centerHx, and the Y-direction position information of the H-th target position information is denoted as centerHy. The acquisition time of the target image corresponding to the H-th target position information can be denoted as tH, where H is a positive integer greater than or equal to 1 and less than or equal to N.
[0109] in,
[0110] DVx=((centerNx-centerMx) / (tN-tM)-(centerMx-center1x) / (tM-t1)) / (tN-tM);
[0111] DVy=((centerNy-centerMy) / (tN-tM)-(centerMy-center1y) / (tM-t1)) / (tN-tM);
[0112] M is the smallest positive integer greater than or equal to N / 2.
[0113] For example, when N is 3,
[0114] DVx=((center3x-center2x) / (t3-t2)-(center2x-center1x) / (t2-t1)) / (t3-t2);
[0115] DVy=((center3y-center2y) / (t3-t2)-(center2y-center1y) / (t2-t1)) / (t3-t2).
[0116] For another example, when N is 4,
[0117] DVx=((center4x-center2x) / (t4-t2)-(center2x-center1x) / (t2-t1)) / (t4-t2);
[0118] DVy=((center4y-center2y) / (t4-t2)-(center2y-center1y) / (t2-t1)) / (t4-t2).
[0119] In addition, the acceleration information and position information can also be divided into three directions, for example, the first direction (X), the second direction (Y) and the third direction (Z), and then the corresponding partial acceleration is calculated according to the position information in each direction. The calculation method of the partial acceleration in a single direction can refer to the above-mentioned implementation methods and will not be elaborated on here.
[0120] It should be noted that, in addition to determining the acceleration information in the above-mentioned manner, the acceleration information may also be determined in other manners, which are not limited thereto.
[0121] In some embodiments,
[0122] The first direction can be denoted as the X direction, and the second direction can be denoted as the Y direction. The component acceleration in the X direction is denoted as DVx, and the component velocity in the Y direction is denoted as DVy. The X-direction position information of the H-th target position information among the N target position information is denoted as centerHx, and the Y-direction position information of the H-th target position information is denoted as centerHy. The acquisition time of the target image corresponding to the H-th target position information can be denoted as tH, where H is a positive integer greater than or equal to 1 and less than or equal to N.
[0123] in,
[0124] DVx=((centerNx-center(N-1)x) / (tN-t(N-1))-(center(N-1)x-center(N-2)x) / (t(N-1)-t(N-2))) / (tN-t(N-1));
[0125] DVy=((centerNy-center(N-1)y) / (tN-t(N-1))-(center(N-1)y-center(N-2)y) / (t(N-1)-t(N-2))) / (tN-t(N-1)).
[0126] For example, when N is 3,
[0127] DVx=((center3x-center2x) / (t3-t2)-(center2x-center1x) / (t2-t1)) / (t3-t2);
[0128] DVy=((center3y-center2y) / (t3-t2)-(center2y-center1y) / (t2-t1)) / (t3-t2).
[0129] For another example, when N is 4,
[0130] DVx=((center4x-center3x) / (t4-t3)-(center3x-center2x) / (t3-t2)) / (t4-t3);
[0131] DVy=((center4y-center3y) / (t4-t3)-(center3y-center2y) / (t3-t2)) / (t4-t3).
[0132] In step S230 , after the speed information and the acceleration information are determined, the motion trajectory can be predicted based on the determined speed information and acceleration information.
[0133] In some embodiments,
[0134] The current moment can be recorded as cur_time. In the position information of the monitored object at the current moment, the position information in the first direction (X) can be recorded as Cx, and the position information in the second direction (Y) can be recorded as Cy. The component velocity in the first direction is recorded as Vx, and the component velocity in the second direction is recorded as Vy. The component acceleration in the first direction is recorded as DVx, and the component velocity in the second direction is recorded as DVy. The X-direction position information of the H-th target position information among the N target position information is recorded as centerHx, and the Y-direction position information of the H-th target position information is recorded as centerHy. The acquisition time of the target image corresponding to the H-th target position information can be recorded as tH, where H is a positive integer greater than or equal to 1 and less than or equal to N.
[0135] in,
[0136] Cx=centerNx+Vx*(cur_time-tN)+(cur_time-tN)*(cur_time-tN)*(DVx / 2);
[0137] Cy=centerNy+Vy*(cur_time-tN)+(cur_time-tN)*(cur_time-tN)*(DVy / 2).
[0138] In addition, the position information can also be divided into three directions, for example, the first direction (X), the second direction (Y) and the third direction (Z), and then the corresponding component velocity and component acceleration are calculated respectively through the position information of each direction, and the position of the direction at the current moment is predicted through the corresponding component velocity and component acceleration. The specific method of determining the position in each direction can refer to the above embodiment, which will not be repeated here.
[0139] It should be noted that, in addition to obtaining the position information through the above-mentioned method, the position information can also be obtained through other methods, which are not limited to this. After determining the position information of any moment from the acquisition moment of the Nth target image to the current moment through the above-mentioned method, the motion trajectory of the monitored object between the acquisition moment of the Nth target image and the current moment can be predicted. When the monitored object is a person, this method can obtain the trajectory of the human coordinates between the acquisition moment of the Nth target image and the current moment. The trajectory of the human coordinates in the monitoring video can be referred to Figure 6 This method can be applied to scenes such as indoors, streets, forests, and farms, without limitation.
[0140] In an exemplary embodiment, a monitoring device is provided, which is applied to a monitoring device. The device can be used to implement the above-mentioned monitoring method. For example, referring to Figure 3 As shown, the device may include:
[0141] The image acquisition module 10 is used to collect and store image data of the monitored object after the image acquisition module is started;
[0142] The AI detection module 20 is configured to, after the AI detection module is started, obtain a target image set based on a startup completion time of the AI detection module and a timestamp of a first frame of the stored image data, where the target image set includes N target images selected from the image data; where N is a positive integer greater than or equal to 3;
[0143] The AI detection module 20 is further configured to process the target image set to obtain a motion trajectory of the monitored object between the capture time of the Nth target image and the current time.
[0144] In an exemplary embodiment, a monitoring device is provided, which is applied to a monitoring device. Figure 3 As shown, in the device, the AI detection module 20 can be used to:
[0145] The startup time difference between the image acquisition module and the AI detection module is obtained by the startup completion moment and the timestamp of the first frame image;
[0146] Split the startup time difference into N-1 time intervals;
[0147] Based on N-1 time intervals, N images are selected from the stored image data as N target images.
[0148] In an exemplary embodiment, a monitoring device is provided, which is applied to a monitoring device. Figure 3 As shown, in the device, the AI detection module 20 can be used to:
[0149] Divide the startup time difference evenly to obtain N-1 equal time intervals.
[0150] In an exemplary embodiment, a monitoring device is provided, which is applied to a monitoring device. Figure 3 As shown, in the device, the AI detection module 20 can be used to:
[0151] Processing the target image to obtain target position information corresponding to the target image; wherein the target position information is the position information of the monitored object at the time of acquisition of the target image;
[0152] Through the N target position information corresponding to the target image set, the speed information and acceleration information of the monitored object corresponding to the Nth target image are obtained;
[0153] The motion trajectory is obtained through velocity information and acceleration information.
[0154] In an exemplary embodiment, a monitoring device is provided, which is applied to a monitoring device. Figure 3 As shown, in the device, the speed information includes a first component speed in a first direction and a second component speed in a second direction, and the first direction and the second direction are perpendicular to each other. The AI detection module 20 can be used to:
[0155] Obtaining a first distance in the first direction and a second distance in the second direction using the Mth target position information and the Nth target position information; wherein M is a minimum positive integer greater than or equal to N / 2;
[0156] Obtain a first component speed through the first distance, the timestamp of the M-th target image, and the timestamp of the N-th target image;
[0157] A second component speed is obtained by using the second distance, the timestamp of the Mth target image, and the timestamp of the Nth target image.
[0158] In an exemplary embodiment, a monitoring device is provided, which is applied to a monitoring device. Figure 3 As shown, in the device, the acceleration information includes a first component acceleration in a first direction and a second component acceleration in a second direction, where the first direction and the second direction are perpendicular to each other. The AI detection module 20 can be used to:
[0159] Obtaining a first distance in the first direction and a second distance in the second direction using the Mth target position information and the Nth target position information; wherein M is a minimum positive integer greater than or equal to N / 2;
[0160] Obtaining a third distance in the first direction and a fourth distance in the second direction through the first target position information and the M-th target position information;
[0161] Obtain a first partial acceleration through the first distance, the third distance, the timestamp of the first target image, the timestamp of the Mth target image, and the timestamp of the Nth target image;
[0162] A second partial acceleration is obtained through the second distance, the fourth distance, the timestamp of the first target image, the timestamp of the Mth target image, and the timestamp of the Nth target image.
[0163] In an exemplary embodiment, a monitoring device is provided, which is applied to a monitoring device. Figure 3 As shown, the apparatus may include an adjustment module 30, which may be used to:
[0164] After obtaining the motion trajectory of the monitored object between the Nth target image and the current moment, the position of the monitoring camera is adjusted based on the motion trajectory; wherein the monitoring camera is used to monitor the monitored object.
[0165] In one exemplary embodiment, a monitoring device is provided. The monitoring device may be a monitoring device equipped with a battery-powered camera, such as a conventional battery-powered camera or a pan-tilt battery-powered camera. The monitoring device may also be other monitoring devices with a camera function, without limitation.
[0166] refer to Figure 4 As shown, the monitoring device 100 may include: at least one processor 101, a memory 102, at least one network interface 104, and another user interface 103. The various components in the monitoring device 100 are coupled together via a bus system 105. It will be appreciated that the bus system 105 is used to enable communication between these components. In addition to a data bus, the bus system 105 also includes a power bus, a control bus, and a status signal bus. However, for clarity, all of these buses will be labeled as the bus system 105.
[0167] The user interface 103 may include a display, a keyboard, or a click monitoring device (eg, a mouse, a trackball, a touch pad, or a touch screen).
[0168] It is understood that the memory 102 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 102 described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0169] In some embodiments, the memory 102 stores the following elements, executable units, or data structures, or a subset thereof, or an extended set thereof: an operating system 1021 and application programs 1022 .
[0170] Among them, the operating system 1021 includes various system programs, such as the framework layer, the core library layer, and the driver layer, which are used to implement various basic services and process hardware-based tasks. The application 1022 includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. The program implementing the method of the embodiment of the application can be included in the application 1022.
[0171] In the embodiment of the present application, the processor 101 is used to execute the methods provided in each method embodiment by calling the program or instructions stored in the memory 102, specifically, the program or instructions stored in the application 1022.
[0172] The methods disclosed in the above embodiments of the present application can be applied to or implemented by processor 101. Processor 101 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 101 or by software instructions. The above processor 101 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software units in the decoding processor. The software units can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 102 , and the processor 101 reads the information in the memory 102 and implements the above method in combination with its hardware.
[0173] It is understood that the embodiments described herein may be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or at least one application-specific integrated circuit (ASIC), digital signal processor (DSP), digital signal processing monitoring device (DSPDevice, DSPD), programmable logic monitoring device (PLD), field-programmable gate array (FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing the functions described herein, or a combination thereof.
[0174] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.
[0175] The present application also provides a storage medium (computer-readable storage medium). The storage medium stores one or at least one program. The storage medium may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; and the memory may also include a combination of the aforementioned types of memory.
[0176] When one or at least one program in a storage medium is executable by one or at least one processor, the storage medium, when used in a monitoring device, can implement the aforementioned method for executing the monitoring device. The processor is configured to execute the monitoring device control program stored in the memory to implement the aforementioned method for executing the monitoring device.
[0177] Professionals should also be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0178] It should be noted that references in this specification to "one embodiment," "an embodiment," "an exemplary embodiment," "some embodiments," and the like indicate that the described embodiments may include a particular feature, structure, or characteristic, but not necessarily every embodiment includes that particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is within the knowledge of those skilled in the art to implement such feature, structure, or characteristic in conjunction with other embodiments, whether explicitly described or not.
[0179] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or monitoring device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or monitoring device. In the absence of more limitations, an element defined by the sentence "comprises a..." does not exclude the presence of other identical elements in the process, method, article or monitoring device that includes the element.
[0180] The above embodiments are only preferred embodiments for fully illustrating the present application, and the protection scope of the present application is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art based on the present application are within the protection scope of the present application.
Claims
1. A monitoring method, characterized in that: The monitoring method comprises: After the image acquisition module is started, the image data of the monitored object is collected and stored by the image acquisition module; After the AI detection module is started, based on the startup completion time of the AI detection module and the timestamp of the first frame of the stored image data, a target image set is obtained, where the target image set includes N target images selected from the image data; where N is a positive integer greater than or equal to 3; The target image set is processed by the AI detection module to obtain the motion trajectory of the monitored object between the acquisition time of the Nth target image and the current time.
2. The monitoring method according to claim 1, characterized in that: The acquiring of a target image set based on the startup completion time of the AI detection module and the timestamp of the first frame image in the stored image data includes: Obtaining a startup time difference between the image acquisition module and the AI detection module through the startup completion moment and the timestamp of the first frame image; Split the startup time difference into N-1 time intervals; Based on N-1 time intervals, N images are selected from the stored image data as the N target images.
3. The monitoring method according to claim 2, characterized in that: The time interval for splitting the startup time difference into N-1 parts includes: The startup time difference is evenly divided to obtain N-1 equal time intervals.
4. The monitoring method according to claim 1, characterized in that: The processing of the target image set by the AI detection module to obtain the motion trajectory of the monitored object between the acquisition time of the Nth target image and the current time includes: Processing the target image through the AI detection module to obtain target position information corresponding to the target image; wherein the target position information is the position information of the monitored object at the time of acquisition of the target image; Obtaining the speed information and acceleration information of the monitored object corresponding to the Nth target image through the N target position information corresponding to the target image set; The motion trajectory is obtained through the velocity information and the acceleration information.
5. The monitoring method according to claim 4, characterized in that: The speed information includes a first component speed in a first direction and a second component speed in a second direction, wherein the first direction and the second direction are perpendicular to each other; The obtaining, through the N target position information corresponding to the target image set, the speed information and acceleration information of the monitored object corresponding to the Nth target image, includes: Obtaining a first distance in the first direction and a second distance in the second direction using the Mth target position information and the Nth target position information; wherein M is a minimum positive integer greater than or equal to N / 2; Obtaining the first component velocity by using the first distance, a timestamp of the Mth target image, and a timestamp of the Nth target image; The second component speed is obtained by using the second distance, the timestamp of the Mth target image, and the timestamp of the Nth target image.
6. The monitoring method according to claim 4, characterized in that: The acceleration information includes a first component acceleration in a first direction and a second component acceleration in a second direction, wherein the first direction and the second direction are perpendicular to each other; The obtaining of the speed information and acceleration information of the monitored object corresponding to the Nth target image by the N target position information corresponding to the target image set includes: Obtaining a first distance in the first direction and a second distance in the second direction using the Mth target position information and the Nth target position information; wherein M is a minimum positive integer greater than or equal to N / 2; Obtaining a third distance in the first direction and a fourth distance in the second direction through the first target position information and the M-th target position information; Obtaining the first component acceleration through the first distance, the third distance, a timestamp of the first target image, a timestamp of the Mth target image, and a timestamp of the Nth target image; The second component acceleration is obtained by using the second distance, the fourth distance, the timestamp of the first target image, the timestamp of the Mth target image, and the timestamp of the Nth target image.
7. The monitoring method according to any one of claims 1 to 6, characterized in that: After obtaining the motion trajectory of the monitored object between the acquisition time of the Nth target image and the current time, the monitoring method includes: Based on the motion trajectory, the position of a surveillance camera is adjusted; wherein the surveillance camera is used to monitor the surveillance object.
8. A monitoring device, characterized in that: The monitoring device comprises: An image acquisition module, configured to acquire and store image data of a monitored object after the image acquisition module is started; an AI detection module, configured to, after the AI detection module is started, acquire a target image set based on a startup completion time of the AI detection module and a timestamp of a first frame of the stored image data, the target image set comprising N target images selected from the image data; wherein N is a positive integer greater than or equal to 3; The AI detection module is further used to process the target image set to obtain the motion trajectory of the monitored object between the acquisition time of the Nth target image and the current time.
9. A monitoring device, characterized in that: The monitoring equipment includes: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the monitoring method according to any one of claims 1 to 7.
10. A storage medium, characterized in that: The storage medium stores one or at least one program, and the one or at least one program can be executed by one or at least one processor to implement the monitoring method according to any one of claims 1 to 7.