A method, system, device and electronic equipment for monitoring exercise intensity

Abstract

This application provides a method, system, device, and electronic device for motion intensity monitoring, relating to the field of image processing technology. The method involves: identifying a target object to be monitored and determining its baseline features; extracting target features from each image to be analyzed regarding the monitored area, and using these target features to determine image regions in the analyzed image whose image features match the baseline features; determining the various motion positions of the target object within the monitored area using the location of each determined image region in its respective image; generating a motion heatmap of the target object based on the first occurrence of the target object at each motion position, and determining the motion intensity of the target object within the acquisition time of the analyzed image based on the motion heatmap. Compared with related technologies, the solution provided in this application can monitor the motion intensity of a target object.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, system, device and electronic device for motion intensity monitoring. Background Technology

[0002] Typically, when the state of a target object changes, the intensity of its motion also changes accordingly. Therefore, by monitoring the intensity of the target object's motion, its state can be understood, allowing for timely detection and appropriate measures to be taken when the target object is in an abnormal state.

[0003] For example, a sudden increase in an infant's activity level indicates that the infant has woken up and needs parental care. Healthy elderly people typically have a consistent daily activity level; if their activity level decreases significantly on a particular day, it's necessary to determine if there's a health issue. Similarly, pets like cats and dogs generally have fixed activity areas and activity levels; changes in their activity levels require monitoring their health. The activity level of livestock such as cattle and sheep on farms also indicates their health; monitoring changes in their activity levels allows for timely assessment of their health, enabling prompt treatment and minimizing potential losses.

[0004] In related technologies, a temperature sensor is installed in a designated area, and then the detection data of the temperature sensor is used to determine a temperature distribution map of the target object. Furthermore, the motion intensity of the target object is determined using the aforementioned temperature distribution map.

[0005] However, in related technologies, the temperature sensor can accurately detect the temperature of the target object only when the target object is close to the temperature sensor. When the target object is far away from the temperature sensor, the target object cannot be tracked, and thus the motion data of the target object cannot be monitored. Consequently, the accuracy of the determined motion intensity of the target object is poor. Summary of the Invention

[0006] The purpose of this application is to provide a method, system, device, and electronic device for monitoring motion intensity, so as to monitor the motion intensity of a target object. The specific technical solution is as follows:

[0007] In a first aspect, embodiments of this application provide a method for monitoring exercise intensity, the method comprising:

[0008] The target object to be monitored is identified, and the baseline features of the target object are determined; wherein the baseline features are determined based on the acquired images of the target object;

[0009] For each image to be analyzed concerning the area to be monitored, target features of the image to be analyzed are extracted, and the target features are used to determine the image region in the image to be analyzed whose image features match the benchmark features;

[0010] By utilizing the location of each determined image region within the image to be analyzed, the movement positions of the target object within the monitored region are determined.

[0011] Based on the first occurrence of the target object at each movement position, a motion heatmap of the target object is generated, and based on the motion heatmap, the motion intensity of the target object is determined within the acquisition time of the image to be analyzed.

[0012] Optionally, in one specific implementation, extracting the target features of each image to be analyzed for the region to be monitored includes:

[0013] For each image to be analyzed concerning the target object, a preset sliding window and sliding hyperparameters are used to determine each window image in the image to be analyzed, and the image features of each window image are extracted as each target feature of the image to be analyzed;

[0014] or,

[0015] For each image to be analyzed concerning the target object, the image to be analyzed is input into a preset feature extraction network to obtain the target features of the image to be analyzed output by the feature extraction network.

[0016] Optionally, in one specific implementation, the target features of the image to be analyzed include: the image features of each extracted window image;

[0017] The step of using the target features to determine the image region in the image to be analyzed that matches the benchmark features includes:

[0018] For each window image, the similarity between the image feature and the reference feature is calculated; and the window image with the maximum similarity and the maximum similarity being greater than a preset similarity is determined as the image region whose image features match the reference feature.

[0019] Optionally, in one specific implementation, the target features of the image to be analyzed include: the target features output by the feature extraction network;

[0020] The step of using the target features to determine the image region in the image to be analyzed that matches the benchmark features includes:

[0021] Using a preset convolution algorithm, a convolution response map is calculated for the target feature and the reference feature. The region in the convolution response map that has the maximum response value and that the maximum response value is greater than a preset response value is determined as the image region whose image features match the reference feature.

[0022] Optionally, in one specific implementation, determining the motion intensity of the target object within the acquisition time of the image to be analyzed, based on the motion heatmap, includes:

[0023] Based on the motion heatmap, the ratio of the area of ​​the sub-region where the first occurrence of the target object is greater than a first preset number of times to the total area of ​​the area to be monitored is calculated as the motion intensity of the target object during the acquisition time of the image to be analyzed.

[0024] Optionally, in one specific implementation, the method further includes:

[0025] Based on the acquisition time of each image to be analyzed and the determined motion positions, the time when the target object appears at each motion position is determined;

[0026] Based on the time at which the target object appears at each movement position, determine the second number of times the target object appears at each movement position within a specified time range during the acquisition time.

[0027] Based on the motion heatmap, the area of ​​the sub-region where the target object appears more than a second preset number of times is calculated as the ratio of the area of ​​the sub-region to the total area of ​​the monitored area, which is used as the specified motion intensity of the target object within the specified time range.

[0028] If the specified motion intensity does not match the historical motion intensity within the specified time range during the historical acquisition period, a notification message regarding the abnormal motion of the target object will be output.

[0029] Optionally, in one specific implementation, determining the baseline features of the target object includes:

[0030] A first type of feature and / or a second type of feature of the target object are determined; and the first type of feature and / or the second type of feature are determined as the reference features of the target object; wherein, the first type of feature is the image feature of the initial region where the target object is located in the first frame of each of the acquired images of the region to be monitored; the second type of feature is the image feature obtained by feature extraction of each of the acquired reference images of the target object in different poses.

[0031] Optionally, in one specific implementation, the method for determining the first type of feature includes:

[0032] Acquire the collected images of the area to be monitored, and determine the initial area where the target object is located in the first frame of each image;

[0033] Feature extraction is performed on the target object in the initial region to obtain the first type of features of the target object;

[0034] The method further includes;

[0035] Each image other than the first frame image is identified as an image to be analyzed.

[0036] Using the initial position, the motion position of the target object in the monitored area is determined when the first frame image is acquired.

[0037] Optionally, in one specific implementation, the method for determining the second type of feature includes:

[0038] Acquire reference images of the target object in different poses;

[0039] Feature extraction is performed on the target object in each reference image to obtain each of the second type of features of the target object.

[0040] Secondly, embodiments of this application provide an exercise intensity monitoring system, the system comprising:

[0041] An image acquisition device is used to acquire images of a target object;

[0042] A processor is configured to execute the steps of any of the embodiments of the motion intensity monitoring method described in the first aspect of this application.

[0043] Thirdly, embodiments of this application provide an exercise intensity monitoring device, the device comprising:

[0044] A feature determination module is used to determine the target object to be monitored and to determine the baseline features of the target object; wherein the baseline features are determined based on the acquired images of the target object;

[0045] The region determination module is used to extract target features from each image to be analyzed for a region to be monitored, and to use the target features to determine the image region in the image to be analyzed whose image features match the benchmark features.

[0046] The position determination module is used to determine the various motion positions of the target object in the monitored area by utilizing the determined regional position of each image region in the image to be analyzed;

[0047] The motion intensity determination module is used to generate a motion heatmap of the target object based on the first occurrence of the target object at each motion position, and to determine the motion intensity of the target object within the acquisition time of the image to be analyzed based on the motion heatmap.

[0048] Optionally, in one specific implementation, the feature determination module is specifically used for:

[0049] For each image to be analyzed concerning the target object, a preset sliding window and sliding hyperparameters are used to determine each window image in the image to be analyzed, and the image features of each window image are extracted as each target feature of the image to be analyzed;

[0050] or,

[0051] For each image to be analyzed concerning the target object, the image to be analyzed is input into a preset feature extraction network to obtain the target features of the image to be analyzed output by the feature extraction network.

[0052] Optionally, in one specific implementation, the target features of the image to be analyzed include: the image features of each extracted window image;

[0053] The region determination module is specifically used for:

[0054] For each window image, the similarity between the image feature and the reference feature is calculated; and the window image with the maximum similarity and the maximum similarity being greater than a preset similarity is determined as the image region whose image features match the reference feature.

[0055] Optionally, in one specific implementation, the target features of the image to be analyzed include: the target features output by the feature extraction network;

[0056] The region determination module is specifically used for:

[0057] Using a preset convolution algorithm, a convolution response map is calculated for the target feature and the reference feature. The region in the convolution response map that has the maximum response value and that the maximum response value is greater than a preset response value is determined as the image region whose image features match the reference feature.

[0058] Optionally, in one specific implementation, the motion intensity determination module is specifically used for:

[0059] Based on the motion heatmap, the ratio of the area of ​​the sub-region where the first occurrence of the target object is greater than a first preset number of times to the total area of ​​the area to be monitored is calculated as the motion intensity of the target object during the acquisition time of the image to be analyzed.

[0060] Optionally, in one specific implementation, the apparatus further includes:

[0061] The time determination module is used to determine the time when the target object appears at each motion position based on the acquisition time of each image to be analyzed and the determined motion positions.

[0062] The second frequency determination module is used to determine, based on the time at which the target object appears at each movement position, the second frequency of the target object appearing at each movement position within a specified time range during the acquisition time.

[0063] The specified motion intensity determination module is used to calculate, based on the motion heatmap, the ratio of the area of ​​the sub-region where the target object appears more than a second preset number of times to the total area of ​​the area to be monitored, as the specified motion intensity of the target object within the specified time range;

[0064] The output module is used to output a notification message about the abnormal movement of the target object if the specified motion intensity does not match the historical motion intensity within the specified time range during the historical acquisition time.

[0065] Optionally, in one specific implementation, the feature determination module is specifically used for:

[0066] A first type of feature and / or a second type of feature of the target object are determined; and the first type of feature and / or the second type of feature are determined as the reference features of the target object; wherein, the first type of feature is the image feature of the initial region where the target object is located in the first frame of each of the acquired images of the region to be monitored; the second type of feature is the image feature obtained by feature extraction of each of the acquired reference images of the target object in different poses.

[0067] Optionally, in one specific implementation, the method for determining the first type of feature includes:

[0068] Acquire the collected images of the area to be monitored, and determine the initial area where the target object is located in the first frame of each image;

[0069] Feature extraction is performed on the target object in the initial region to obtain the first type of features of the target object;

[0070] The device also includes;

[0071] The image to be analyzed determination module is used to determine each image other than the first frame image as an image to be analyzed.

[0072] The motion position determination module is used to determine the motion position of the target object in the monitored area when the first frame image is acquired, using the initial position.

[0073] Optionally, in one specific implementation, the method for determining the second type of feature includes:

[0074] Acquire reference images of the target object in different poses;

[0075] Feature extraction is performed on the target object in each reference image to obtain each of the second type of features of the target object.

[0076] Fourthly, embodiments of this application provide an electronic device, including:

[0077] Memory, used to store computer programs;

[0078] When a processor executes a program stored in memory, it implements the steps of any of the above method embodiments.

[0079] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above method embodiments.

[0080] Sixthly, embodiments of this application also provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps of any of the above method embodiments.

[0081] Beneficial effects of the embodiments in this application:

[0082] As can be seen from the above, by applying the solution provided in the embodiments of this application, the target object to be monitored can be determined first, and the baseline features of the target object can be determined based on the acquired images of the target object; then, for each image to be analyzed about the area to be monitored, the target features of the image to be analyzed can be extracted, and the image regions in the image to be analyzed that have image features that match the baseline features can be determined using the region position of each determined image region in the image to be analyzed; then, the movement positions of the target object in the area to be monitored can be determined using the region position of each determined image region in the image to be analyzed. In this way, a motion heat map of the target object can be generated based on the first occurrence of the target object at each movement position, and then, based on the motion heat map, the motion intensity of the target object during the acquisition time of the image to be analyzed can be determined.

[0083] Based on this, the solution provided in this application can track the motion position of the target object based on the baseline features of the object to be monitored and the target features of each image to be analyzed, thereby obtaining the motion intensity of the target object within the acquisition time. Since the tracking of the target object's motion position is based on the baseline features of the target object and the target features of each image to be analyzed, it eliminates the need for temperature sensor detection data. Therefore, it avoids the situation in related technologies where the target object cannot be tracked when it is far away, and also avoids the situation where the target object's motion data cannot be monitored due to its distance. Thus, compared to methods that track objects by detecting the temperature of the target object, it can more accurately monitor the motion intensity of the target object. Furthermore, when there are multiple target objects in the image to be analyzed, the target features of each image to be analyzed can be compared with the baseline features of each target object, thereby improving the accuracy of identifying each target object. In this way, after tracking the motion position of each target object, the motion intensity of each target object can be determined, thereby achieving the purpose of monitoring the motion intensity of multiple target objects.

[0084] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0085] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0086] Figure 1 This is a schematic flowchart of a method for monitoring exercise intensity provided in an embodiment of this application;

[0087] Figure 2 Another schematic flowchart of the exercise intensity monitoring method provided in this application embodiment;

[0088] Figure 3 This is a specific example diagram of the motion thermogram provided in the embodiments of this application;

[0089] Figure 4 This is another schematic flowchart of the exercise intensity monitoring method provided in the embodiments of this application;

[0090] Figure 5 A schematic diagram of the various window images in the image to be analyzed provided in the embodiments of this application;

[0091] Figure 6 A schematic diagram of a convolutional response map provided in an embodiment of this application;

[0092] Figure 7 A flowchart illustrating the process of determining the various movement positions reached by the object to be monitored, provided in an embodiment of this application;

[0093] Figure 8 A flowchart illustrating a specific example of an exercise intensity monitoring method provided in this application embodiment;

[0094] Figure 9 This is a schematic diagram of the structure of a sports intensity monitoring system provided in an embodiment of this application;

[0095] Figure 10 This is a schematic diagram of the structure of a sports intensity monitoring device provided in an embodiment of this application;

[0096] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0097] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0098] In the technical solution of this application, the operations of obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0099] In related technologies, a temperature sensor is installed in a designated area to determine a temperature distribution map of a target object, and the motion intensity of the target object is determined using this temperature distribution map. However, in these related technologies, the temperature sensor can only accurately detect the temperature of the target object when it is close to the sensor. When the target object is far from the sensor, it cannot be tracked, and therefore its motion data cannot be monitored, resulting in poor accuracy in determining the motion intensity of the target object.

[0100] To address the aforementioned technical problems, this application provides a method for monitoring exercise intensity.

[0101] This method can be applied to various application scenarios that require monitoring the movement intensity of the target, such as monitoring the movement intensity of the elderly in the home; monitoring the movement intensity of the animals raised in a farm; and monitoring the movement intensity of vehicles passing through a certain section of road.

[0102] Furthermore, this method can be applied to various image acquisition devices, such as cameras, video cameras, and surveillance equipment. These image acquisition devices can be equipped with image processing modules, and then execute this method to process the acquired images and determine the motion intensity of the object to be monitored. It can also be applied to other electronic devices that can communicate with the image acquisition device, such as servers, mobile phones, and computers. These electronic devices can acquire the images acquired by the image acquisition device and then execute this method to determine the motion intensity of the object to be monitored.

[0103] Therefore, the embodiments of this application do not specifically limit the application scenarios and execution entities of the method.

[0104] An exercise intensity monitoring method provided in this application embodiment may include the following steps:

[0105] The target object to be monitored is identified, and the baseline features of the target object are determined; wherein the baseline features are determined based on the acquired images of the target object;

[0106] For each image to be analyzed concerning the area to be monitored, target features of the image to be analyzed are extracted, and the target features are used to determine the image region in the image to be analyzed whose image features match the benchmark features;

[0107] By utilizing the location of each determined image region within the image to be analyzed, the movement positions of the target object within the monitored region are determined.

[0108] Based on the first occurrence of the target object at each movement position, a motion heatmap of the target object is generated, and based on the motion heatmap, the motion intensity of the target object is determined within the acquisition time of the image to be analyzed.

[0109] As can be seen from the above, by applying the solution provided in the embodiments of this application, the target object to be monitored can be determined first, and the baseline features of the target object can be determined based on the acquired images of the target object; then, for each image to be analyzed about the area to be monitored, the target features of the image to be analyzed can be extracted, and the image regions in the image to be analyzed that have image features that match the baseline features can be determined using the region position of each determined image region in the image to be analyzed; then, the movement positions of the target object in the area to be monitored can be determined using the region position of each determined image region in the image to be analyzed. In this way, a motion heat map of the target object can be generated based on the first occurrence of the target object at each movement position, and then, based on the motion heat map, the motion intensity of the target object during the acquisition time of the image to be analyzed can be determined.

[0110] Based on this, the solution provided in this application can track the motion position of the target object based on the baseline features of the object to be monitored and the target features of each image to be analyzed, thereby obtaining the motion intensity of the target object within the acquisition time. Since the tracking of the target object's motion position is based on the baseline features of the target object and the target features of each image to be analyzed, it eliminates the need for temperature sensor detection data. Therefore, it avoids the situation in related technologies where the target object cannot be tracked when it is far away, and also avoids the situation where the target object's motion data cannot be monitored due to its distance. Thus, compared to methods that track objects by detecting the temperature of the target object, it can more accurately monitor the motion intensity of the target object. Furthermore, when there are multiple target objects in the image to be analyzed, the target features of each image to be analyzed can be compared with the baseline features of each target object, thereby improving the accuracy of identifying each target object. In this way, after tracking the motion position of each target object, the motion intensity of each target object can be determined, thereby achieving the purpose of monitoring the motion intensity of multiple target objects.

[0111] The following, with reference to the accompanying drawings, provides a detailed description of an exercise intensity monitoring method provided in an embodiment of this application.

[0112] Figure 1 This is a flowchart illustrating a method for monitoring exercise intensity provided in an embodiment of this application, as shown below. Figure 1 As shown, the method may include the following steps S101-S104.

[0113] S101: Determine the target object to be monitored and determine the baseline characteristics of the target object;

[0114] Among them, the baseline features are determined based on the acquired images of the target object;

[0115] To monitor the motion intensity of various objects within a monitored area, image acquisition devices can be installed to capture images of that area. For example, surveillance equipment, cameras, or other image acquisition devices can be installed to capture images of the monitored area. In this way, images of the monitored area captured by the image acquisition devices can be obtained, and then the motion intensity of each object within that monitored area can be determined using these images.

[0116] Therefore, in order to monitor the motion intensity of an object, the target object to be monitored can be identified first, and its baseline characteristics can be determined. Then, the motion intensity of the target object can be determined by using the images of the area to be monitored acquired by the image acquisition device, thus achieving the purpose of monitoring the motion intensity of the target object.

[0117] The aforementioned benchmark features are obtained by extracting features from various images of the target object under different motion states. The extracted features of the target object under different motion states can be used to identify the target object.

[0118] Furthermore, the aforementioned benchmark features can be HOG (Histogram of Oriented Gradient) features, SIFT (Scale-Invariant Feature Transform) features, deep learning features, or other features; all of these are reasonable and are not specifically limited in this embodiment. Correspondingly, the method for extracting the aforementioned benchmark features can be HOG, SIFT, deep learning methods, or other methods; all of these are reasonable and are not specifically limited in this embodiment.

[0119] The target object can be at least one object pre-determined among various objects located in the area to be monitored. For example, the target object can be one object located in the area to be monitored, or it can be all objects located in the area to be monitored. Correspondingly, the reference features of the target object can be features pre-determined based on various acquired images of the target object. In this way, by acquiring various images to be analyzed about the area to be monitored, the motion intensity of the target object can be monitored. Furthermore, when all objects located in the area to be monitored are taken as the target objects to be monitored, the motion intensity of each target object can be monitored separately.

[0120] The target object mentioned above can also be at least one object selected from among the various objects included in the specified image of the area to be monitored. For example, object A in the specified image can be selected as the target object, or each object in the specified image can be selected as the target object. Accordingly, the reference features of the target object can be the features obtained by extracting features from the image region where the target object is located in the specified image.

[0121] Optionally, in one specific implementation, step S101, determining the baseline features of the target object, may include the following step 11:

[0122] Step 11: Determine the first type of features and / or the second type of features of the target object; and determine the first type of features and / or the second type of features as the baseline features of the target object.

[0123] The first type of feature is the image feature of the initial region where the target object is located in the first frame of each image of the area to be monitored; the second type of feature is the image feature obtained by extracting features from each reference image of the target object in different poses.

[0124] In this specific implementation, a first type of feature of the target object can be determined, and thus the first type of feature is used as the baseline feature of the target object; or a second feature of the target object can be determined, and thus the second type of feature is used as the baseline feature of the target object; or a first type of feature and a second type of feature of the target object can be determined, and thus the first type of feature and the second type of feature are used as the baseline feature of the target object.

[0125] The first type of feature is based on the image features of the initial region where the target object is located in the first frame of each image of the area to be monitored; the second type of feature is the image features obtained by extracting features from each reference image of the target object in different poses.

[0126] Optionally, in one specific implementation, the determination of the aforementioned first type of feature may include the following steps 21-22:

[0127] Step 21: Acquire the collected images of the area to be monitored, and determine the initial area where the target object is located in the first frame of each image;

[0128] Step 22: Extract features from the target object in the initial region to obtain the first type of features of the target object;

[0129] Therefore, the exercise intensity monitoring method provided in this application embodiment may further include the following steps 31-32;

[0130] Step 31: Identify each image other than the first frame as the image to be analyzed;

[0131] Step 32: Using the initial position, determine the motion position of the target object in the area to be monitored when the first frame image is acquired.

[0132] In this specific implementation, images of the area to be monitored are acquired by an image acquisition device. Then, the target object is selected from the first frame of these images, and the initial region where the target object is located is determined within that first frame. Feature extraction is then performed on this initial region to obtain the first type of features of the target object.

[0133] Among them, the first frame image mentioned above is the earliest image acquired among all the images of the area to be monitored;

[0134] In some cases, the first frame of the acquired images may not include the target object to be monitored, or the imaging quality of the first frame may be poor, making it impossible to identify the target object in the first frame. In such cases, an earlier acquired image that includes the target object can be used as the first frame, and the target object and its initial region can be determined within this first frame.

[0135] Optionally, multiple objects included in the first frame image are identified, and the target object to be monitored is selected from among the multiple objects;

[0136] Optionally, based on the user's selection instruction, the object indicated by the selection instruction in the first frame image is taken as the target object to be monitored.

[0137] Then, using the initial position described above, the movement position of the target object in the area to be monitored can be determined when the first frame image was acquired. Next, each frame acquired, excluding the first frame image, can be designated as an image to be analyzed regarding the area to be monitored, and the first type of feature can be used as the reference feature of the target object. Thus, the image region where the target object is located can be determined in each of the images to be analyzed using the reference feature.

[0138] Based on steps 21-22 above, as follows Figure 2 As shown in the figure, this application embodiment also provides a method for monitoring exercise intensity, which may include the following steps S201-S206:

[0139] S201: Acquire the collected images of the area to be monitored, determine the target object to be monitored in the first frame of each image, and determine the initial area where the target object is located in the first frame of the image;

[0140] S202: Extract features from the target object in the initial region to obtain the first type of features of the target object, and use the first type of features as the baseline features of the target object;

[0141] S203: Identify each image other than the first frame image as the image to be analyzed, and use the initial position to determine the movement position of the target object in the monitoring area when the first frame image is acquired;

[0142] S204: For each image to be analyzed concerning the area to be monitored, extract the target features of the image to be analyzed, and use the target features to determine the image region in the image to be analyzed whose image features match the baseline features;

[0143] S205: Using the location of each determined image region in the image to be analyzed, determine the movement positions of the target object in the monitored area;

[0144] S206: Generate a motion heatmap of the target object based on the first occurrence of the target object at each motion position, and determine the motion intensity of the target object within the acquisition time of the image to be analyzed based on the motion heatmap.

[0145] Since steps S201-S203 are implemented in the same way as steps 21-22 and 31-32, they will not be described again here. After obtaining the first type of image features of the target object, these first type of features can be used as the reference features of the target object. In this way, the image region where the target object is located can be determined in each image to be analyzed using the reference features of the target object. Steps S204-S206 are implemented in the same way as steps S102-S104, and will be explained later.

[0146] Optionally, in one specific implementation, the determination of the second type of feature mentioned above may include the following steps 41-42:

[0147] Step 41: Acquire reference images of the target object in different poses;

[0148] Step 42: Extract features from the target object in each reference image to obtain the second type of features of the target object.

[0149] In this specific implementation, reference images of the target object in different poses can be acquired. Then, feature extraction can be performed on the target object in each reference image to obtain the second type of features of the target object.

[0150] Among these, the different postures of the target object can be different sides of the target object, such as the front of the target object, the back of the target object, etc.; it can also be the target object in different motion states such as walking, running, jumping, squatting, etc.; or it can be that the various local features of the target object are in different forms, such as the target object's hand being in an open state, the target object's hand being in a clenched fist state, etc.

[0151] In this way, the acquired images of the area to be monitored can be used as images to be analyzed, and the second-type features of the target object can be used as the baseline features of the target object. Then, the image region where the target object is located can be determined in the images to be analyzed using the baseline features.

[0152] Based on steps 21-22 and 41-42 above, the first and second types of features of the target object can also be used as the baseline features of the target object.

[0153] Optionally, using the first and second types of features of the target object as the baseline features of the target object may include the following steps 51-53:

[0154] Step 51: Acquire the collected images of the area to be monitored, and determine the initial area where the target object is located in the first frame of each image. Extract features from the target object in the initial area to obtain the first type of features of the target object.

[0155] Step 52: Control the target object to present different postures, acquire the base images of the target object in different postures, extract features from the target object in each base image, and obtain the second type of features of the target object.

[0156] Step 53: Use the first type of features and the second type of features mentioned above as the baseline features of the target object.

[0157] Step 51 is implemented in the same way as steps 21-22 above, and step 52 is implemented in the same way as steps 41-42 above, so it will not be described again here.

[0158] After determining the first and second types of features of the target object, these features can be used as the baseline features of the target object. This allows us to determine the target object's movement position within the monitored area when the first frame of the image is acquired, using the initial position of the target object. Subsequently, each frame acquired, excluding the first frame, is designated as an image to be analyzed. Then, using the baseline features, the image region containing the target object can be determined within each of these images to be analyzed.

[0159] S102: For each image to be analyzed concerning the area to be monitored, extract the target features of the image to be analyzed, and use the target features to determine the image region in the image to be analyzed whose image features match the baseline features;

[0160] To analyze the motion intensity of the target object, various images of the monitored area can be obtained.

[0161] Optionally, the video of the area to be monitored, acquired by the image acquisition device, can be used to analyze the video, and then each image to be analyzed can be obtained from the video.

[0162] For each image to be analyzed concerning the area to be monitored, the target features of the image to be analyzed can be extracted. Then, the target features of the target object can be used to determine the image region in the image to be analyzed that matches the aforementioned benchmark features.

[0163] Specifically, for each image to be analyzed, the target features of the image to be analyzed can be matched with the reference features of the target object, thereby determining the image features in the target features that match the reference features, and then determining the image region in the image to be analyzed that has image features that match the reference features.

[0164] The aforementioned extraction of the target features of the image to be analyzed can be either extracting the features of the entire image or extracting the features of each image region in the image to be analyzed. Both are reasonable and are not specifically limited in the embodiments of this application.

[0165] Furthermore, the matching of the above image features with the reference features means that the image features have a high degree of similarity with the reference features. Generally, the image features of the same object in different images have a high degree of similarity. Therefore, for two different images, the target features of the two images are extracted respectively. If there are image features with a high degree of similarity among the two extracted target features, it can be considered that the object corresponding to the image feature in the two images is the same object.

[0166] Based on this, the image region in the image to be analyzed that has image features that match the baseline features can be considered as the image region where the target object is located in the image to be analyzed.

[0167] Of course, since there are situations such as the target object moving outside the monitoring area or the target object not appearing in the image to be analyzed, there may be no image region in the image to be analyzed that has image features that match the baseline features. Therefore, it can be determined that the target object does not exist in the image to be analyzed, and the next image to be analyzed can be continued.

[0168] For clarity, the specific implementation of step S102 will be explained later.

[0169] S103: Using the location of each determined image region in the image to be analyzed, determine the movement positions of the target object in the monitored area;

[0170] Since each image to be analyzed is an image of the area to be monitored acquired by an image acquisition device, and for the same image acquisition device, the transformation relationship between the image coordinate system and the world coordinate system is fixed. After determining the location of the target object in the image to be analyzed, the image coordinates of the target object's location in the image coordinate system can be determined. Furthermore, based on the aforementioned transformation relationship, the world coordinates of the target object's location in the world coordinate system can be determined. Since the image acquisition device acquires a spatial image of the area to be monitored, the determined world coordinates of the target object can be considered as the spatial coordinates of the target object within the area to be monitored. Therefore, based on the spatial coordinates of the target object, the movement position of the target object within the area to be monitored can be determined.

[0171] S104: Generate a motion heatmap of the target object based on the first occurrence of the target object at each motion position, and determine the motion intensity of the target object within the acquisition time of the image to be analyzed based on the motion heatmap.

[0172] Based on the acquisition time of each image to be analyzed and the movement position of the target object in the area to be monitored, the first occurrence of the target object at each movement position can be determined. Furthermore, based on these first occurrences, a motion heatmap of the target object can be generated.

[0173] Specifically, for each movement position, the first occurrence count of the target object at that position is the sum of the total number of times the target object appears at that position. For example, if the target object is located at position A in the first frame, then the first occurrence count at position A is 1; if the target object is also located at position A in the fourth frame, then the first occurrence count at position A is updated to 2. After analyzing N frames of images, the first occurrence count of the target object at position A can be obtained.

[0174] Optionally, the above-mentioned motion heat map can be a curve graph about the target object, which can more intuitively determine the first occurrence of the target object at each motion position;

[0175] Optionally, the aforementioned motion heatmap can be a location diagram showing the specific location of each motion position within the area to be monitored, and each motion position is marked with the first number of times the target object appears at that position; for example, such as Figure 3 As shown, the motion heat map marks the first appearance of the target object at each motion position. The target object appeared 860 times at motion position 310 and 46 times at motion position 320. The number of times the target object appeared at other motion positions has been marked on the map and will not be listed here.

[0176] Optionally, the above-mentioned motion heat map can be a location diagram of the specific location of each motion position within the area to be monitored, and each motion position is marked with the first number of times the target object appears at that motion position, as well as the specific time of each appearance at that position;

[0177] Optionally, the above-mentioned motion heat map can be a trajectory map of the target object, with each trajectory point marked with the time corresponding to that trajectory point and the location number of the target object's movement position at that time.

[0178] Since this motion heatmap is based on the first occurrence count of the target object at each motion position, the larger the first occurrence count of the target object at a certain motion position, the larger the value of that motion position in the motion heatmap can be. If there are multiple motion positions in the above motion heatmap with first occurrence counts greater than a preset threshold, that is, the sum of the areas of the sub-regions where the first occurrence counts of the motion positions with first occurrence counts greater than the preset threshold are large, it can be determined that the target object has reached many motion positions and the target object's activity range is relatively wide, thus the target object's motion intensity is high. Conversely, if there is only one motion position in the above motion heatmap with first occurrence counts greater than the preset threshold, that is, the sum of the areas of the sub-regions where the first occurrence counts of the motion positions with first occurrence counts greater than the preset threshold are very small, it can be determined that the target object has reached very few motion positions and the target object only stays at that one motion position, thus the target object's motion intensity is low.

[0179] Based on this, the motion intensity of the target object during the acquisition time of each image to be analyzed can be determined according to the above motion heat map.

[0180] Specifically, the motion intensity of the target object during the acquisition time of each image to be analyzed can be calculated by using the motion positions in the motion heatmap where the number of times the target object appears is greater than a preset first preset number; alternatively, the brightness of each region in the motion heatmap can be calculated using the first preset number, and then the motion intensity of the target object during the acquisition time of each image to be analyzed can be calculated using the brightness of each region.

[0181] Optionally, in one specific implementation, step S104 above, which determines the motion intensity of the target object within the acquisition time of the image to be analyzed based on the motion heatmap, may include the following step 61:

[0182] Step 61: Based on the motion heatmap, calculate the ratio of the area of ​​the sub-region where the first occurrence of the target object is greater than the first preset number of occurrences to the total area of ​​the area to be monitored, and use this ratio as the motion intensity of the target object during the acquisition time of the image to be analyzed.

[0183] In this specific implementation, after generating a motion heatmap of the target object, the sub-regions where the first occurrence of the target object is greater than a first preset number of times can be determined based on the motion heatmap. Then, the sum of the areas of the sub-regions is calculated, and the ratio of the sum of the areas to the total area of ​​the area to be monitored is calculated. In this way, the ratio can be used as the motion intensity of the target object during the acquisition time of each image to be analyzed.

[0184] Wherein, the first preset number is not greater than the sum of the first occurrences of the target object in each sub-region; and the first preset number can be set according to actual needs, such as 10 times, 80 times, etc., which are all reasonable and are not specifically limited in this application embodiment.

[0185] Optionally, the above-mentioned kinetic thermodynamics can be used. Figure 2 Value-based mapping involves marking each sub-region in the aforementioned motion heatmap where the first count is greater than a preset threshold as 1, and marking each sub-region in the aforementioned motion heatmap where the first count is not greater than the preset threshold as 0. In this way, the aforementioned motion heatmap can be converted into a binary image. Therefore, the ratio of the area of ​​each sub-region marked as 1 in the binary image to the total area of ​​the area to be monitored can be used as the motion intensity of the target object during the acquisition time of each image to be analyzed.

[0186] After calculating the motion intensity of the target object within the acquisition time of each image to be analyzed, the motion intensity of the target object can be compared and analyzed with the historical motion intensity of the target object to determine whether there are any abnormalities in the motion of the target object.

[0187] Alternatively, in one specific implementation, such as Figure 4 As shown, the exercise intensity monitoring method provided in this application embodiment further includes the following steps S105-S108:

[0188] S105: Based on the acquisition time of each image to be analyzed and the determined motion positions, determine the time when the target object appears at each motion position;

[0189] S106: Based on the time when the target object appears at each movement position, determine the second number of times the target object appears at each movement position within a specified time range during the acquisition time.

[0190] S107: Based on the motion heat map, calculate the ratio of the area of ​​the sub-region where the target object appears more than the second preset number of times to the total area of ​​the area to be monitored, and use it as the specified motion intensity of the target object within the specified time range;

[0191] S108: If the specified motion intensity does not match the historical motion intensity within the specified time range of the historical acquisition time, output a notification message about the abnormal motion of the target object.

[0192] In this specific implementation, for each image to be analyzed, the acquisition time of the image and the movement position of the target object in the monitored area at the time of image acquisition can be determined. Then, based on each image to be analyzed, the number of times the target object appears at each movement position and the time of each appearance can be determined. Thus, based on the number of times the target object appears at each movement position and the time of each appearance, the second number of times the target object appears at each movement position within a specified time range during the acquisition time can be determined.

[0193] Then, based on the above motion heat map, the motion location where the second number is greater than the second preset number can be determined, and the ratio of the area of ​​the sub-region where the second number is greater than the second preset number is located to the total area of ​​the area to be monitored can be calculated. Then, the above ratio is used as the specified motion intensity of the target object within a specified time range.

[0194] The specified time range is determined based on the time period corresponding to the above-mentioned heat map. For example, if the time range corresponding to the above-mentioned heat map is 8:00 AM to 8:00 PM on Saturday, then the specified time range can be 8:00 AM to 10:00 AM on Saturday, or 2:00 PM to 5:00 PM on Saturday, or 8:00 AM to 8:00 PM on Saturday.

[0195] Furthermore, the specified motion intensity can be analyzed against the historical motion intensity within a specified time range during the historical data collection period. If the specified motion intensity does not match the historical motion intensity within the specified time range during the historical data collection period, a notification message about the abnormal motion of the target object can be output to alert relevant personnel that the motion of the target object is abnormal. If the specified motion intensity matches the historical motion intensity within the specified time range during the historical data collection period, the specified motion intensity can be archived, and the motion intensity of the target object can continue to be monitored.

[0196] Wherein, the second preset number is not greater than the sum of the second number of times the target object appears in each sub-region; and the second preset number can be set according to actual needs, such as 10 times, 80 times, etc., which are all reasonable and are not specifically limited in this application embodiment.

[0197] Furthermore, based on the aforementioned motion heatmap, the motion trajectory of the target object can be determined. The aforementioned historical motion intensity can include the motion intensity of the target object in each time period and the motion trajectory of the target object in that time period. In this way, the determined motion intensity of the target object can be compared with the historical motion intensity to determine whether there is any abnormality in the motion state of the target object. The so-called abnormality in the motion state of the target object can refer to a significant difference between the motion intensity of the target object within a specified time range and the historical motion intensity; it can also refer to a significant difference between the motion position of the target object within a specified time period and the historical motion intensity; or it can refer to a significant difference between the motion trajectory of the target object within a specified time period and the historical motion trajectory. These are all reasonable and are not specifically limited in this embodiment.

[0198] For example, in a home setting, historical exercise intensity data shows that an elderly person walks in the yard every morning between 9 and 10 a.m. If the exercise intensity of the elderly person is monitored and it is determined that the exercise intensity of the elderly person in the yard between 9 and 10 a.m. on a given day is 0, indicating an abnormal exercise pattern, a prompt message can be output to alert the elderly person to the abnormal exercise pattern.

[0199] As can be seen from the above, the solution provided in this application can track the motion position of the target object based on the baseline features of the object to be monitored and the target features of each image to be analyzed, thereby obtaining the motion intensity of the target object within the acquisition time. Since the tracking of the target object's motion position is based on the baseline features of the target object and the target features of each image to be analyzed, it eliminates the need for temperature sensor detection data. Therefore, it avoids the situation in related technologies where the target object cannot be tracked when it is far away, and also avoids the situation where the target object's motion data cannot be monitored due to its distance. Thus, compared to methods that track objects by detecting the temperature of the target object, it can more accurately monitor the motion intensity of the target object. Furthermore, when there are multiple target objects in the image to be analyzed, the target features of each image to be analyzed can be compared with the baseline features of each target object, thereby improving the accuracy of identifying each target object. In this way, after tracking the motion position of each target object, the motion intensity of each target object can be determined, thereby achieving the purpose of monitoring the motion intensity of multiple target objects.

[0200] The specific implementation of step S102 above will be explained below.

[0201] The target features of the image to be analyzed can be extracted using a preset feature extraction method. Furthermore, the target features of the image to be analyzed will be different when different feature extraction methods are used. Consequently, the methods for determining the image region in the image to be analyzed that matches the reference features will also be different.

[0202] Optionally, in one specific implementation, step S102 above, extracting the target features of each image to be analyzed for the region to be monitored, may include the following step 71:

[0203] Step 71: For each image to be analyzed concerning the target object, use a preset sliding window and sliding hyperparameters to determine each window image in the image to be analyzed, and extract the image features of each window image as each target feature of the image to be analyzed.

[0204] In this specific implementation, for each image to be analyzed concerning the target object, a preset sliding window and sliding hyperparameters can be used to determine each window image of the image to be analyzed. Then, the image features of each window image can be extracted, and the extracted image features can be used as the target features of the image to be analyzed.

[0205] The aforementioned preset sliding window can be a single sliding window or multiple sliding windows. For each sliding window, the aforementioned sliding hyperparameters can include the window shape, window size, sliding direction, and sliding step size, etc., and the aforementioned window shape, window size, sliding direction, and sliding step size can all be set according to actual needs.

[0206] The window shape can be rectangular, trapezoidal, or other shapes.

[0207] The window size mentioned above can be in pixels, for example, 30 pixels * 40 pixels, 50 pixels * 80 pixels, etc.;

[0208] The sliding direction can be from a specified position in the image to a specified direction. For example, starting from the top left corner of the image, slide down first, then slide to the right after sliding down one sliding step; or starting from the bottom right corner of the image, slide up first, then slide to the left when sliding to the top right corner of the image.

[0209] The aforementioned sliding step size can be a specified number of pixels, such as 10 pixels, 15 pixels, etc.; and the sliding step size can be different when the preset sliding window slides in different sliding directions. For example, when the preset sliding window slides down from the top left corner of the image, the sliding step size can be 10 pixels, and when the preset sliding window slides to the right from the top left corner of the image, the sliding step size can be 15 pixels. These are all reasonable and are not specifically limited in the embodiments of this application.

[0210] For example, such as Figure 5 As shown, the preset sliding window includes rectangular sliding windows 510 and 520 of different sizes. By sliding the two rectangular sliding windows respectively, multiple window images can be obtained. Then, by extracting the image features of each window image, the target features of the image to be analyzed can be obtained.

[0211] Optionally, in one specific implementation, step S102 above, extracting the target features of each image to be analyzed for the region to be monitored, may include the following step 81:

[0212] Step 81: For each image to be analyzed concerning the target object, input the image to be analyzed into a preset feature extraction network to obtain the target features of the image to be analyzed output by the feature extraction network.

[0213] In this specific implementation, for each image to be analyzed concerning the target object, the image to be analyzed can be input into a preset feature extraction network, thereby obtaining the target features of the image to be analyzed output by the feature extraction network.

[0214] When different methods are used to extract the target features of the image to be analyzed, the target features of the image to be analyzed will be different. Consequently, the methods for determining the image region in the image to be analyzed that matches the reference features will also be different.

[0215] Optionally, in one specific implementation, the target features of the image to be analyzed include: the image features of each extracted window image;

[0216] Step S102 above, which uses target features to determine the image region in the image to be analyzed whose image features match the reference features, may include the following steps 91-92:

[0217] Step 91: For each window image, calculate the similarity between the image features and the baseline features;

[0218] Step 92: The window image with the highest similarity and the highest similarity is greater than the preset similarity is determined as the image region whose image features match the baseline features.

[0219] In this specific implementation, the target features of the image to be analyzed may include the image features of each extracted window image. For each window image's image features, the similarity between the image features and the baseline features of the target object can be calculated. Furthermore, the window image with the highest similarity, and whose highest similarity is greater than a preset similarity, can be determined as the image region whose image features match the baseline features.

[0220] The aforementioned preset similarity can be set according to actual needs, such as 60% or 90%, which are all reasonable and are not specifically limited in this embodiment.

[0221] Furthermore, the image region that matches the benchmark features can characterize the image region with the benchmark features, that is, it can characterize the image region where the target object is located.

[0222] Optionally, during the sliding process of the aforementioned preset sliding window, when obtaining each window image, the image features of each obtained window image are extracted, and the similarity between the image features and the reference features of the target object is calculated. Then, after obtaining the similarity scores for each window image, the maximum similarity among the various similarities is determined, and it is determined whether the maximum similarity is greater than a preset similarity. If the maximum similarity is greater than the preset similarity, the window image with the maximum similarity can be considered as the image region whose image features match the reference features; if the maximum similarity is less than the preset similarity, it can be determined that there is no image region in the image to be analyzed whose image features match the reference features.

[0223] Optionally, after the aforementioned sliding window process is completed and each window image is obtained, image features of each window image can be extracted, and the similarity between each image feature and the baseline features of the target object can be calculated. Then, the maximum similarity among the obtained similarities can be determined, and it can be determined whether the maximum similarity is greater than a preset similarity. If the maximum similarity is greater than the preset similarity, the window image with the maximum similarity can be considered as the image region whose image features match the baseline features; if the maximum similarity is less than the preset similarity, it can be determined that there is no image region in the image to be analyzed whose image features match the baseline features.

[0224] For example, such as Figure 5 As shown, Figure 5 The region 530 within the dashed box is the image region whose image features match the baseline features.

[0225] Optionally, in one specific implementation, the target features of the image to be analyzed include: target features output by the feature extraction network;

[0226] Step S102 above, which uses target features to determine the image region in the image to be analyzed whose image features match the reference features, may include the following step 10A:

[0227] Step 10A: Using a preset convolution algorithm, calculate the convolution response map of the target feature and the reference feature, and determine the region in the convolution response map that has the maximum response value and the maximum response value is greater than the preset response value as the image region whose image features match the reference feature.

[0228] In this specific implementation, the target features of the image to be analyzed may include the target features output by the feature extraction network. For each image to be analyzed, a preset convolution algorithm can be used to calculate the convolution response map of the target features of the image to be analyzed and the reference features of the target object. The region in the convolution response map that has the maximum response value and the maximum response value is greater than the preset response value is determined as the image region whose image features match the reference features.

[0229] In this embodiment, when the aforementioned feature extraction network is used to extract the target features and the baseline features of the target object in the image to be analyzed, the target features and baseline features with an M*N*N structure can be output. These target features and baseline features can then be input into a convolutional network to output the convolutional response maps of the target features and baseline features. Here, M represents the number of convolutional channels, and N*N represents the size of the convolutional kernel. Both M and N can be set according to actual needs; for example, M can be 32, 64, etc., and N can be 5, 7, etc., which are all reasonable and are not specifically limited in this embodiment.

[0230] The aforementioned preset response values ​​can be set according to actual needs, such as 10, 40, etc., which are all reasonable and are not specifically limited in this application embodiment.

[0231] In other words, when obtaining the target features of each image to be analyzed, these target features and the baseline features of the target object can be input into a convolutional network to obtain a convolutional response map of the target features and the baseline features. Furthermore, the response values ​​of each region in this response map can be obtained. This allows us to determine the maximum response value among all response values ​​and further determine whether the maximum response value is greater than a preset response value. If the maximum response value is greater than the preset response value, the region in the convolutional response map with the maximum response value can be identified as the image region whose image features match the baseline features. If the maximum response value is not greater than the preset response value, it can be determined that there is no image region in the image to be analyzed whose image features match the baseline features.

[0232] For example, Figure 6 The convolutional response maps of the target features and reference features of the image to be analyzed are shown below. Figure 6 As shown, the area with the highest brightness in the middle of the image is the area with the maximum response value, which is greater than the preset response value. In other words, it is the image area in the image to be analyzed whose image features match the baseline features.

[0233] To facilitate understanding of the exercise intensity monitoring method provided in the embodiments of this application, the following is combined with... Figure 7 and Figure 8 A specific example is provided below. The object to be monitored in this example is the target object in the embodiments of this application, and the method for monitoring the motion intensity of the object to be monitored may include the following steps S801-S806:

[0234] S801: Determine whether the image is the first frame of the video; if yes, proceed to step S802; otherwise, proceed to step S803.

[0235] S802: Feature extraction yields either Feat1 or DFeat1;

[0236] S803: Feature extraction yields Feats or Dfeats;

[0237] S804: Feature matching or convolution yields the response map;

[0238] S805: Obtain the location of the object to be monitored;

[0239] S806: Statistically analyze the trajectory of the object to be monitored, calculate the motion intensity, and output the motion intensity.

[0240] like Figure 7 As shown, to monitor the motion intensity of the object to be monitored within the monitored area, a video of the monitored area is first acquired, and then divided into multiple frames. Each frame is then inspected to determine the image region where the object to be monitored is located. Using the image regions where the object is located in each frame, the coordinate values ​​corresponding to each image region are determined, and multiple coordinate values ​​of the object within the monitored area are output. These multiple coordinate values ​​represent the various motion positions reached by the object.

[0241] When analyzing each frame of the image to be analyzed, it can be first determined whether the frame is the first frame of the video; if so, the baseline features Feat1 or DFeat1 of the object to be monitored can be extracted based on the frame. Feat1 is the second type of feature of the object to be monitored obtained by extracting features from the region where the object to be monitored is located in the frame; DFeat1 is a set of the second type of feature of the object to be monitored and the first type of feature of the object to be monitored under different postures, extracted beforehand.

[0242] If the image frame is not the first frame, feature extraction can be performed on it. The sliding window method can be used to extract the image features of each window of the image to be analyzed, obtaining the target features (Feats) of the image frame; alternatively, the image frame can be input into a preset feature extraction network to obtain the target features (Dfeats).

[0243] Then, feature matching or convolution is performed on the above-mentioned baseline features and target features to obtain the image region in the frame to be analyzed that has image features that match the above-mentioned baseline features. Then, based on the above-mentioned image features, the motion positions of the object to be monitored in the monitoring region can be obtained. Then, the trajectory of the object to be monitored can be counted to generate the motion heat map of the object to be monitored. Using the above-mentioned motion heat map, the motion intensity of the object to be monitored can be calculated and output.

[0244] Based on the same inventive concept, and corresponding to the embodiments provided in this application above... Figure 1 The present application provides a method for monitoring exercise intensity, and also provides an exercise intensity monitoring system.

[0245] Figure 9 This is a schematic diagram of the structure of a motion intensity monitoring system provided in an embodiment of this application, as shown below. Figure 9 As shown, the system may include an image acquisition unit 100 and a processor 200.

[0246] The image acquisition unit 100 is used to acquire images of the target object.

[0247] The processor 200 is used to execute a motion intensity monitoring method provided in the embodiments of this application.

[0248] In this specific implementation, in order to monitor the motion intensity of the target object, the image acquisition unit 100 can acquire an image of the target object, and then send the acquired image to the processor 200. In this way, the processor 200 can execute a motion intensity monitoring method provided in this application embodiment based on the image to determine the motion intensity of the target object.

[0249] The specific implementation of the exercise intensity monitoring method provided in this application embodiment has been described in detail above. The processor 200 executes each step of the exercise intensity monitoring method provided in this application embodiment according to the specific implementation described above, and will not be repeated here.

[0250] Furthermore, based on the same inventive concept, and corresponding to the embodiments provided in this application above... Figure 1 The present application provides a method for monitoring exercise intensity, and also provides an exercise intensity monitoring device.

[0251] Figure 10 This is a schematic diagram of the structure of a motion intensity monitoring device provided in an embodiment of this application, as shown below. Figure 10 As shown, the device may include the following modules:

[0252] The feature determination module 1010 is used to determine the target object to be monitored and to determine the baseline features of the target object; wherein the baseline features are determined based on the acquired images of the target object;

[0253] The region determination module 1020 is used to extract target features of each image to be analyzed for a region to be monitored, and to use the target features to determine the image region in the image to be analyzed whose image features match the reference features.

[0254] The position determination module 1030 is used to determine the various motion positions of the target object in the monitored area by utilizing the determined regional position of each image region in the image to be analyzed.

[0255] The motion intensity determination module 1040 is used to generate a motion heat map of the target object based on the first occurrence of the target object at each motion position, and to determine the motion intensity of the target object within the acquisition time of the image to be analyzed based on the motion heat map.

[0256] As can be seen from the above, the solution provided in this application can track the motion position of the target object based on the baseline features of the object to be monitored and the target features of each image to be analyzed, thereby obtaining the motion intensity of the target object within the acquisition time. Since the tracking of the target object's motion position is based on the baseline features of the target object and the target features of each image to be analyzed, it eliminates the need for temperature sensor detection data. Therefore, it avoids the situation in related technologies where the target object cannot be tracked when it is far away, and also avoids the situation where the target object's motion data cannot be monitored due to its distance. Thus, compared to methods that track objects by detecting the temperature of the target object, it can more accurately monitor the motion intensity of the target object. Furthermore, when there are multiple target objects in the image to be analyzed, the target features of each image to be analyzed can be compared with the baseline features of each target object, thereby improving the accuracy of identifying each target object. In this way, after tracking the motion position of each target object, the motion intensity of each target object can be determined, thereby achieving the purpose of monitoring the motion intensity of multiple target objects.

[0257] Optionally, in one specific implementation, the feature determination module 1010 is specifically used for:

[0258] For each image to be analyzed concerning the target object, a preset sliding window and sliding hyperparameters are used to determine each window image in the image to be analyzed, and the image features of each window image are extracted as each target feature of the image to be analyzed;

[0259] or,

[0260] For each image to be analyzed concerning the target object, the image to be analyzed is input into a preset feature extraction network to obtain the target features of the image to be analyzed output by the feature extraction network.

[0261] Optionally, in one specific implementation, the target features of the image to be analyzed include: the image features of each extracted window image;

[0262] The region determination module 1020 is specifically used for:

[0263] For each window image, the similarity between the image feature and the reference feature is calculated; and the window image with the maximum similarity and the maximum similarity being greater than a preset similarity is determined as the image region whose image features match the reference feature.

[0264] Optionally, in one specific implementation, the target features of the image to be analyzed include: the target features output by the feature extraction network;

[0265] The region determination module 1020 is specifically used for:

[0266] Using a preset convolution algorithm, a convolution response map is calculated for the target feature and the reference feature. The region in the convolution response map that has the maximum response value and that the maximum response value is greater than a preset response value is determined as the image region whose image features match the reference feature.

[0267] Optionally, in one specific implementation, the motion intensity determination module 1040 is specifically used for:

[0268] Based on the motion heatmap, the ratio of the area of ​​the sub-region where the first occurrence of the target object is greater than a first preset number of times to the total area of ​​the area to be monitored is calculated as the motion intensity of the target object during the acquisition time of the image to be analyzed.

[0269] Optionally, in one specific implementation, the apparatus further includes:

[0270] The time determination module is used to determine the time when the target object appears at each motion position based on the acquisition time of each image to be analyzed and the determined motion positions.

[0271] The second frequency determination module is used to determine, based on the time at which the target object appears at each movement position, the second frequency of the target object appearing at each movement position within a specified time range during the acquisition time.

[0272] The specified motion intensity determination module is used to calculate, based on the motion heatmap, the ratio of the area of ​​the sub-region where the target object appears more than a second preset number of times to the total area of ​​the area to be monitored, as the specified motion intensity of the target object within the specified time range;

[0273] The output module is used to output a notification message about the abnormal movement of the target object if the specified motion intensity does not match the historical motion intensity within the specified time range during the historical acquisition time.

[0274] Optionally, in one specific implementation, the feature determination module 1010 is specifically used for:

[0275] A first type of feature and / or a second type of feature of the target object are determined; and the first type of feature and / or the second type of feature are determined as the reference features of the target object; wherein, the first type of feature is the image feature of the initial region where the target object is located in the first frame of each of the acquired images of the region to be monitored; the second type of feature is the image feature obtained by feature extraction of each of the acquired reference images of the target object in different poses.

[0276] Optionally, in one specific implementation, the method for determining the first type of feature includes:

[0277] Acquire the collected images of the area to be monitored, and determine the initial area where the target object is located in the first frame of each image;

[0278] Feature extraction is performed on the target object in the initial region to obtain the first type of features of the target object;

[0279] The device also includes;

[0280] The image to be analyzed determination module is used to determine each image other than the first frame image as an image to be analyzed.

[0281] The motion position determination module is used to determine the motion position of the target object in the monitored area when the first frame image is acquired, using the initial position.

[0282] Optionally, in one specific implementation, the method for determining the second type of feature includes:

[0283] Acquire reference images of the target object in different poses;

[0284] Feature extraction is performed on the target object in each reference image to obtain each of the second type of features of the target object.

[0285] This application also provides an electronic device, such as... Figure 11 As shown, it includes:

[0286] Memory 1101 is used to store computer programs;

[0287] When the processor 1102 executes the program stored in the memory 1101, it implements the steps of any of the motion intensity monitoring methods provided in the embodiments of this application.

[0288] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 1102, the communication interface, and the memory 1101 communicating with each other via the communication bus.

[0289] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0290] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0291] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0292] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0293] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described motion intensity monitoring methods.

[0294] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the motion intensity monitoring methods described above.

[0295] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0296] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0297] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments, device embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0298] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for monitoring exercise intensity, characterized in that, The method includes: The target object to be monitored is identified, and the baseline features of the target object are determined; wherein the baseline features are determined based on the acquired images of the target object; For each image to be analyzed concerning the area to be monitored, target features of the image to be analyzed are extracted, and the target features are used to determine the image region in the image to be analyzed whose image features match the benchmark features; By utilizing the location of each determined image region within the image to be analyzed, the movement positions of the target object within the monitored region are determined. Based on the number of times the target object appears at each movement position, a motion heatmap of the target object is generated. Based on the motion heatmap, the area of ​​the sub-region where the number of times the target object appears for the first time is greater than a first preset number of times is calculated as the ratio of the area of ​​the sub-region where the target object appears for the first time to the total area of ​​the area to be monitored. This ratio is used as the motion intensity of the target object during the acquisition time of the image to be analyzed.

2. The method according to claim 1, characterized in that, For each image to be analyzed regarding the area to be monitored, the extraction of target features from that image includes: For each image to be analyzed concerning the target object, a preset sliding window and sliding hyperparameters are used to determine each window image in the image to be analyzed, and the image features of each window image are extracted as each target feature of the image to be analyzed; or, For each image to be analyzed concerning the target object, the image to be analyzed is input into a preset feature extraction network to obtain the target features of the image to be analyzed output by the feature extraction network.

3. The method according to claim 2, characterized in that, The target features of the image to be analyzed include: the image features extracted from each window image; The step of using the target features to determine the image region in the image to be analyzed that matches the benchmark features includes: For each window image, the similarity between the image feature and the reference feature is calculated; and the window image with the maximum similarity and the maximum similarity being greater than a preset similarity is determined as the image region whose image features match the reference feature.

4. The method according to claim 2, characterized in that, The target features of the image to be analyzed include: the target features output by the feature extraction network; The step of using the target features to determine the image region in the image to be analyzed that matches the benchmark features includes: Using a preset convolution algorithm, a convolution response map is calculated for the target feature and the reference feature. The region in the convolution response map that has the maximum response value and that the maximum response value is greater than a preset response value is determined as the image region whose image features match the reference feature.

5. The method according to claim 1, characterized in that, The method further includes: Based on the acquisition time of each image to be analyzed and the determined motion positions, the time when the target object appears at each motion position is determined; Based on the time at which the target object appears at each movement position, determine the second number of times the target object appears at each movement position within a specified time range during the acquisition time. Based on the motion heatmap, the area of ​​the sub-region where the target object appears more than a second preset number of times is calculated as the ratio of the area of ​​the sub-region to the total area of ​​the monitored area, which is used as the specified motion intensity of the target object within the specified time range. If the specified motion intensity does not match the historical motion intensity within the specified time range during the historical acquisition period, a notification message regarding the abnormal motion of the target object will be output.

6. The method according to any one of claims 1-5, characterized in that, The determination of the baseline features of the target object includes: A first type of feature and / or a second type of feature of the target object are determined; and the first type of feature and / or the second type of feature are determined as the reference features of the target object; wherein, the first type of feature is the image feature of the initial region where the target object is located in the first frame of each of the acquired images of the region to be monitored; the second type of feature is the image feature obtained by feature extraction of each of the acquired reference images of the target object in different poses.

7. The method according to claim 6, characterized in that, The method for determining the first type of feature includes: Acquire the collected images of the area to be monitored, and determine the initial area where the target object is located in the first frame of each image; Feature extraction is performed on the target object in the initial region to obtain the first type of features of the target object; The method further includes; Each image other than the first frame image is identified as an image to be analyzed. Using the initial region, the motion position of the target object in the region to be monitored is determined when the first frame image is acquired.

8. The method according to claim 6, characterized in that, The methods for determining the second type of feature include: Acquire reference images of the target object in different poses; Feature extraction is performed on the target object in each reference image to obtain each of the second type of features of the target object.

9. An exercise intensity monitoring system, the system comprising: An image acquisition device is used to acquire images of a target object; A processor for performing the method according to any one of claims 1-8.

10. A motion intensity monitoring device, characterized in that, The device includes: A feature determination module is used to determine the target object to be monitored and to determine the baseline features of the target object; wherein the baseline features are determined based on the acquired images of the target object; The region determination module is used to extract target features from each image to be analyzed for a region to be monitored, and to use the target features to determine the image region in the image to be analyzed whose image features match the benchmark features. The position determination module is used to determine the various motion positions of the target object in the monitored area by utilizing the determined regional position of each image region in the image to be analyzed; The motion intensity determination module is used to generate a motion heatmap of the target object based on the number of times the target object appears at each motion position, and to calculate the ratio of the area of ​​the sub-region where the number of times the target object appears for the first time is greater than a first preset number of times to the total area of ​​the area to be monitored, based on the motion heatmap, as the motion intensity of the target object during the acquisition time of the image to be analyzed.

11. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-8.

Images