Counting method, apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application provide a counting method, comprising: obtaining a test site image; determining a plurality of to-be-identified regions based on a reference object in the test site image; obtaining a target image set, the target image set comprising a plurality of images of at least one testee performing a movement in at least one to-be-identified region of the plurality of to-be-identified regions; identifying a posture of the testee in each to-be-identified region of each image in the target image set, and determining a number of times that the testee completes the movement in each to-be-identified region. Embodiments of the present application also provide a counting device, an electronic device, and a storage medium.

Description

Technical Field

[0001] This application relates to the field of electronic technology, and in particular to a counting method and apparatus, electronic device and storage medium. Background Technology

[0002] Currently, body composition counting methods are mainly divided into three categories: manual, hardware sensor, and machine vision. Manual methods involve real-time monitoring of each participant, observing their movements, counting their actions, and recording the results after the test. Hardware sensor-based methods require deploying a set of hardware devices at the testing site; participants wear the corresponding hardware, and the system automatically counts their movements in real time. Machine vision-based methods primarily involve segmenting video into frames, detecting key points on the human body, and determining the angles of these key points for real-time counting.

[0003] However, manual body measurement counting methods are very labor-intensive, cannot automatically obtain results, and have low testing efficiency; hardware-based body measurement counting methods have high equipment costs, complex deployment, cannot prevent non-test subjects from taking the test, are easily interfered with, and have poor accuracy; machine vision-based body measurement counting methods also cannot prevent non-test subjects from taking the test, and cannot solve the problems of interference from other people in the picture and multiple people counting simultaneously. Summary of the Invention

[0004] This application provides a counting method, a counting device, an electronic device, and a computer-readable storage medium.

[0005] The technical solution of this application is implemented as follows:

[0006] This application provides a counting method, including:

[0007] Acquire images of the test site;

[0008] Based on reference objects in the test site images, multiple regions to be identified are determined; a target image set is obtained, the target image set including multiple images of at least one test subject moving within at least one region to be identified in the multiple regions to be identified;

[0009] The poses of the test subject within the plurality of regions to be identified in each image of the target image set are identified, and the number of movements completed by the test subject within each region to be identified is determined.

[0010] This application provides a counting device, including:

[0011] The acquisition unit is used to acquire images of the test site;

[0012] The identification unit is used to determine multiple regions to be identified based on reference objects in the test site image; the acquisition unit is also used to acquire a target image set, the target image set including multiple images of at least one test subject moving within at least one region to be identified in the multiple regions to be identified;

[0013] A classification unit is used to perform action recognition on the test subject in the plurality of regions to be identified in each image of the target image set, and to determine the number of times the test subject completes the movement in each region to be identified.

[0014] This application provides an electronic device, including a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the counting method provided above.

[0015] This application provides a computer-readable storage medium for storing a computer program that causes a computer to execute the counting method described above.

[0016] In some embodiments of this application, the technical solutions first involve acquiring a test site image; then, based on a reference object in the test site image, determining multiple regions to be identified; next, acquiring multiple images of at least one test subject moving within at least one region to be identified, thus obtaining a target image set; finally, identifying the posture of the test subject within each image in the target image set, and determining the number of movements completed by the test subject within each region. This refined division of the region by determining multiple regions to be identified allows the test subject to move within the regions to be identified, automatically counting the postures of the movements performed by the test subject within each region, preventing interference from outside the region. Furthermore, having multiple regions to be identified allows for simultaneous testing and automatic counting of multiple test subjects, reducing manpower and material resources and improving the efficiency of physical fitness testing. Attached Figure Description

[0017] Figure 1 A counting method flowchart provided in the embodiments of this application Figure 1 ;

[0018] Figure 2 A counting method flowchart provided in the embodiments of this application Figure 2 ;

[0019] Figure 3 A diagram illustrating an identification model architecture provided in this application embodiment;

[0020] Figure 4 A counting method flowchart provided in the embodiments of this application Figure 3 ;

[0021] Figure 5 A diagram illustrating the architecture of a pose classification model provided in this application embodiment;

[0022] Figure 6 A counting method flowchart provided in the embodiments of this application Figure 4 ;

[0023] Figure 7 A flowchart of the sit-up counting method provided in the embodiments of this application;

[0024] Figure 8a This application provides a schematic diagram of the region to be analyzed by the recognition model.

[0025] Figure 8b The target frame of the reference object is provided for the embodiments of this application;

[0026] Figure 9 This is a schematic diagram of the structural composition of a counting device provided in an embodiment of this application;

[0027] Figure 10 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0029] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and they all fall within the protection scope of the embodiments of this application.

[0030] In addition, in the embodiments of this application, "first," "second," etc. are used to distinguish similar objects, and are not necessarily used for a specific order or sequence.

[0031] In practical applications, while manual body measurement methods can guarantee no cheating, inconsistencies in subjective standards and understanding of the rules among individuals lead to inconsistent results. This is particularly problematic when many participants are involved, as it is extremely labor-intensive, cannot automatically generate scores, and results in low efficiency. Hardware-based body measurement methods often require individual sets of equipment, which are cumbersome, costly, and complex to deploy. They also fail to prevent unauthorized testing and require users to wear certain hardware, impacting test accuracy. Furthermore, infrared sensing measurements are susceptible to interference due to variations in height, weight, and other physiological characteristics among participants, resulting in lower accuracy. Machine vision-based body measurement methods also cannot prevent unauthorized testing and cannot address issues such as interference from other individuals in the frame or simultaneous counting by multiple participants.

[0032] When conducting physical fitness tests, invigilators hope to achieve the principle of fairness and impartiality by automatically counting the physical fitness movements of test subjects with minimal manpower and material resources, while using a unified subjective assessment standard, thereby improving the efficiency of the physical fitness tests.

[0033] Based on the problems existing in the related technologies, this application provides a counting method. The subject executing the counting method can be the counting device provided in this application, or an electronic device that integrates the technology device. The counting device can be implemented in hardware or software.

[0034] Figure 1 This is a flowchart illustrating a counting method provided in an embodiment of this application, as shown below. Figure 1 As shown, the counting method may include:

[0035] Step S110: Obtain the test site image.

[0036] Optionally, the test site image can be obtained through a front-facing or rear-facing camera integrated into an electronic device. The camera can be a monocular, binocular, or tri-lens camera; or it can be provided by a device that is independent of the electronic device but has a shooting function, such as a computer, mobile phone, or tablet. This application embodiment does not limit the method of obtaining the test site image.

[0037] Optionally, the test site images in this application embodiment are taken from various angles and heights, and this application embodiment does not limit the shooting angle and height of the test site.

[0038] In this embodiment of the application, the electronic device obtains the test site image by taking its own picture or by providing the test site image through a third-party device.

[0039] Optionally, the test site image can be an image of the venue where the test subject is being tested, such as a playground, indoor stadium, outdoor stadium, or any other venue where testing can be conducted. This application embodiment does not limit the test site image.

[0040] Step S120: Based on the reference object in the test site image, determine multiple regions to be identified;

[0041] Optionally, the reference object in the test site image is a reference object needed by the test subject when performing the test, such as a mat needed by the test subject for sit-ups, a horizontal bar needed by the test subject for pull-ups, a mat needed by the test subject for push-ups, etc., and the reference object is related to the test subject in the test site.

[0042] Optionally, the number of reference objects in the test site image can be one or more. For example, there can be one or more mats in the test site image when doing sit-ups, one or more horizontal bars in the test site image when doing pull-ups, and one or more mats in the test site image when doing push-ups. This application embodiment does not limit the number of reference objects in the test site image.

[0043] Optionally, the position of the reference object in the test site image can be placed arbitrarily, but the position cannot be changed after placement.

[0044] For example, based on the different positions of multiple reference objects in the test site image, the reference objects in the test site image are divided into preset ranges, and the divided regions are numbered 1, 2, 3, ..., N (N is the total number of reference objects).

[0045] It should be understood that the area to be identified is determined by the electronic device based on reference objects in the test site image. Since the number of reference objects can be one or more, and their positions can be arbitrarily placed, the number of areas to be identified can be one or more based on the number of reference objects, and the position of the area to be identified can be arbitrarily based on the position of the reference objects. In this embodiment of the application, the area to be identified is not determined according to a fixed standard.

[0046] In this embodiment, the electronic device determines multiple regions to be identified based on reference objects in a test site image. One reference object determines one region to be identified, and multiple reference objects determine multiple regions to be identified. By determining multiple regions to be identified, the regions are accurately divided, preventing external interference, and multiple regions allow multiple test subjects to be tested simultaneously.

[0047] Step S130: Obtain a target image set, which includes multiple images of at least one test subject moving within at least one region to be identified in the plurality of regions to be identified.

[0048] After determining multiple regions to be identified, the electronic device acquires multiple images of at least one subject moving within the regions to be identified, thus obtaining a target image set.

[0049] Optionally, the target image set can be acquired by a front-facing camera or a rear-facing camera integrated into an electronic device. The camera can be a monocular camera, a binocular camera, or a tri-lens camera; or it can be provided by a device that is independent of the electronic device but has a shooting function, such as a computer, a mobile phone, a tablet, etc. The embodiments of this application do not limit the method of acquiring the target image set.

[0050] Optionally, the target image set can be a series of consecutive frames of images or a video segment by extracting key frames to reduce video redundancy. This application does not impose specific limitations on the embodiments.

[0051] Optionally, the target image set in this application embodiment is taken from various angles and heights, and this application embodiment does not limit the shooting angle and height of the test site.

[0052] In this embodiment, after determining multiple regions to be identified, the electronic device acquires multiple images of at least one test subject moving within the regions to be identified, thus obtaining a target image set. By acquiring multiple images of the test subject's movement within the regions to be identified, the range of motion of the test subject is obtained, preventing interference from outside the regions to be identified.

[0053] Step S140: Identify the pose of the test subject in the plurality of regions to be identified in each image of the target image set, and determine the number of movements completed by the test subject in each region to be identified.

[0054] In this embodiment of the application, after acquiring the target image set, the electronic device identifies the posture of the test subject in multiple regions to be identified in each image of the target image set, and counts the movements made by the test subject in each region to be identified based on the identification results.

[0055] Optionally, posture refers to the action performed by the test subject when completing a movement. Completing a movement requires continuously completing the starting posture, intermediate posture, and ending posture. For example, when doing sit-ups, there are two postures: supine posture, intermediate posture, and starting posture; when doing pull-ups, there are two postures: the test subject's head is below the bar and the head is above the bar; when doing push-ups, there are two postures: the test subject's waist is straight and the waist is bent forward. In this application, the embodiments do not limit the posture.

[0056] In summary, the electronic device acquires images of the test site, determines multiple regions to be identified based on reference objects within these images, and acquires multiple images, including at least one test subject moving within each region, to obtain a target image set. Multiple regions to be identified within each image in the target image set are then identified, and the number of movements completed by the test subject within each region is determined based on the identification results. By identifying multiple regions to be identified, fine-grained segmentation of the regions is achieved, the range of movement for the test subject is determined, and interference from outside the regions is prevented. Furthermore, multiple regions allow for simultaneous testing of multiple test subjects and automatic counting. This helps reduce the consumption of manpower and resources and improves the efficiency of physical fitness testing.

[0057] It should be noted that step S120, which determines multiple regions to be identified based on reference objects in the test site image, can include directly extracting features from the test site image to obtain a target feature map of the test site image. By performing feature processing on the input test site image, data storage and input data bandwidth are reduced, and redundancy is minimized.

[0058] Feature extraction involves sampling an image using a convolutional network to output feature maps at different stages. Different stages have different resolutions due to varying levels of processing, resulting in lower resolution target feature maps.

[0059] For example, resolution can be an image resolution, typically expressed as the number of pixels in each direction, such as 640×480, 1080×720, etc. The higher the image resolution, the more pixels it contains, and the clearer the image. At least two resolution parameters refer to two different numbers of pixels in the image, such as 4×4 and 2×2.

[0060] Direct feature extraction from test site images results in a low-resolution target feature map. This is detrimental to identifying multiple regions to be recognized based on a reference object within the test site image. Therefore, electronic devices can extract features from the test site image at at least two resolutions to avoid the problem of low target feature map resolution hindering the identification of multiple regions to be recognized.

[0061] Specifically, in one embodiment of this application, reference is made to... Figure 2 As shown, step S120, based on the reference object in the test site image, determines multiple regions to be identified. This can also be achieved in the following ways:

[0062] Step S210: Extract features from the test site image based on at least two resolution parameters to obtain at least two target feature maps associated with the two resolution parameters.

[0063] Step S220: Merge the at least two target feature maps to obtain a merged feature map.

[0064] Step S230: Perform identification processing on the reference object in the merged feature map to determine the region to be identified.

[0065] Among them, at least two resolution parameters can be any two or more resolution parameters among high resolution, medium resolution, and low resolution; or they can be any two or more different resolution parameters. For example, at least two resolutions can be 16×16 and 8×8, or 4×4, 2×2, and 1×1. This application does not limit the resolution parameters.

[0066] In this embodiment, feature extraction is performed by using convolutional layers to extract features such as color, brightness, texture, shape, and geometry from an image.

[0067] In this embodiment of the application, after inputting a test site image, at least two different sampling methods are performed on the test site image to obtain at least two target feature maps, wherein the resolution parameters of the at least two target feature maps are different. The at least two target feature maps are merged to obtain a merged feature map, and the reference object in the merged feature map is identified to determine the region to be identified.

[0068] For example, input a test site image with a resolution of 8×8, perform convolution and deconvolution operations on the test site image using 2×2 to perform upsampling and downsampling by 2 times, and obtain two target feature maps with resolutions of 16×16 and 4×4. Merge the two target feature maps with resolutions of 16×16 and 4×4, and perform recognition processing on the reference object in the merged feature map to determine the region to be recognized.

[0069] Optionally, the size of the input test site image can be preset, such as 16×16, 8×8, 4×4, etc. This application embodiment does not limit the size of the input test site image.

[0070] In one embodiment of this application, step S210 extracts features from the test site based on at least two resolution parameters to obtain at least two target feature maps associated with the two resolution parameters. This can also be achieved in the following ways:

[0071] Step S211: Based on the at least two resolution parameters, sample the test site image to obtain at least two initial images; the at least two initial images correspond one-to-one with the at least two resolution parameters.

[0072] The initial image is obtained by sampling the test site image in at least two different ways, resulting in at least two initial feature maps, i.e., the initial image. Since the initial image is obtained through at least two different sampling methods, it has at least two resolution parameters.

[0073] Example, reference Figure 3 As shown, Figure 3 The high-definition panoramic image viewed from above is the test site image in this embodiment of the application. Different sampling processes are applied to the input test site image to obtain a first-resolution initial image H(k), a second-resolution initial image M(k), and a third-resolution initial image L(k), respectively. The resolution of H(k) is the same as the resolution of the input test site image, the resolution of H(k) is greater than the resolution of M(k), and the resolution of M(k) is greater than the resolution of L(k). k is the number of network layers when the network model is optimal, as determined experimentally.

[0074] Step S212: Based on the at least two resolution parameters, sample each of the at least two initial images to obtain at least two images to be processed corresponding to each initial image.

[0075] In this embodiment of the application, at least two images to be processed are obtained by performing at least two different samplings on each of the at least two initial images, and the resolution parameters of the at least two images to be processed are different.

[0076] Example, reference Figure 3As shown in the processing of layers 1 to 2, different sampling processes are applied to the obtained H(k) to obtain images of different resolutions: a first resolution image, a second resolution image, and a third resolution image. The resolution of the first resolution image is the same as the resolution of H(k), the resolution of the first resolution image is greater than the resolution of the second resolution image, and the resolution of the second resolution image is greater than the resolution of the third resolution image. Similarly, different sampling processes are applied to the obtained M(k) to obtain images of different resolutions: a fourth resolution image, a fifth resolution image, and a sixth resolution image. The resolution of the fourth resolution image is the same as the resolution of H(k), the resolution of the fifth resolution image is the same as the resolution of M(k), and the resolution of the sixth resolution image is the same as the resolution of L(k). Similarly, by performing different sampling processes on L(k), we obtain the seventh-resolution image, the eighth-resolution image, and the ninth-resolution image corresponding to M(k). The seventh-resolution image has the same resolution as H(k), the eighth-resolution image has the same resolution as M(k), and the ninth-resolution image has the same resolution as L(k). It can be understood that after sampling each of the at least two initial images, nine images with three different resolutions are obtained, each corresponding to one of the three initial images.

[0077] Step S213: Extract features from the multiple images to be processed corresponding to the at least two initial images respectively, and merge the image features of the images to be processed with the same resolution parameters to obtain at least two first feature maps; the at least two first feature maps correspond one-to-one with the at least two resolution parameters.

[0078] In this embodiment, feature extraction is performed on each of the multiple images to be processed, and the image features of images with the same resolution are merged to obtain at least two first feature maps. The resolution parameters of the at least two first feature maps are different.

[0079] Example, reference Figure 3 As shown in the processing of the second to third layers, feature extraction is performed on the nine images to be processed corresponding to the three initial images. The image features of the nine images to be processed with the same resolution parameters are merged to obtain three first feature maps, and the three first feature maps correspond one-to-one with the resolution parameters of the initial images.

[0080] Step S214: Continue to sample each of the at least two first feature maps based on the at least two resolution parameters to obtain at least two images to be processed corresponding to each first feature map. Then, merge and extract features from the multiple images to be processed corresponding to the at least two first feature maps to obtain at least two second feature maps. Continue to process the second feature maps until at least two Nth feature maps are obtained; N is an integer greater than 2.

[0081] Continue to sample each feature map in at least two different ways to obtain at least two images to be processed corresponding to each first feature map. Merge and extract features from the images to be processed with the same resolution parameters to obtain at least two second feature maps. Continue to sample the second feature maps until at least two Nth feature maps are obtained.

[0082] refer to Figure 3 The processing between layer 3 and layer k is repeated with the above steps S212 and S213 to obtain the Nth feature map, where N is k / 2 and k is the number of network layers required for the best recognition effect.

[0083] It is understandable that electronic devices can downsample high-resolution images and upsample low-resolution images by transposing convolutions during feature extraction. In this way, information in parallel multi-resolution sub-networks is repeatedly exchanged to improve the accuracy of feature extraction.

[0084] By integrating spatial information at different resolutions and global information at a single resolution, the feature extraction capability of the model for both spatial and global information at different resolutions is guaranteed.

[0085] Step S215: Use the at least two Nth feature maps as the target feature maps.

[0086] In this embodiment of the application, at least two Nth feature maps obtained through repeated processing are used as target feature maps.

[0087] In this embodiment of the application, step S213 extracts features from multiple images to be processed corresponding to the at least two initial images, and merges images of the images to be processed with the same resolution parameters to obtain at least two first feature maps. This can also be achieved in the following ways:

[0088] Step S2131: Divide each feature map of the multiple images to be processed with the same resolution into multiple image blocks of the same size.

[0089] Example, reference Figure 3Each of the nine feature maps to be processed with the same resolution parameters is divided into multiple image blocks of the same size.

[0090] Step S2132: Merge the image blocks corresponding to the feature maps to be processed with the same resolution into an image block sequence; to obtain at least two image block sequences with different resolutions.

[0091] Example, reference Figure 3 The image blocks corresponding to the feature maps to be processed with the same resolution are merged into an image block sequence, resulting in three image block sequences with different resolutions.

[0092] Step S2133: Quantize the image block sequence and add position encoding information to obtain the at least two first feature maps.

[0093] Example, reference Figure 3 Each image block sequence in the three different resolution image block sequences is quantized and the position information of the reference object is added to obtain three first feature maps.

[0094] Positional encoding is a method that uses positional information to represent each pixel in a sequence in a secondary way, allowing the input data to carry positional information so that the model can find positional features.

[0095] In this embodiment, by adding the location information of the reference object to the image block sequence, the model can identify the location of the reference object. By integrating spatial information at different resolutions and global information at a single resolution, the feature extraction capability of the model's spatial and global information at different resolutions is guaranteed, thereby more accurately determining multiple regions to be identified.

[0096] In one embodiment of this application, reference is made to Figure 3 As shown, the identification of the merged feature map and determination of the region to be identified in step S230 can also be achieved in the following ways:

[0097] Based on the merged feature map, the location of the key points of the reference object is determined, and the location of the key points of the reference object is determined as the region to be identified.

[0098] In this embodiment, a merged feature map is obtained after feature extraction from the test site image. The merged feature map is then used for recognition to obtain the output result. The output result includes the center position of the reference object's key points and the deviation between the center position of the reference object's key points and the center position of the actual reference object's key points. A regression loss function is used to continuously reduce the difference between the output center position of the reference object's key points and the actual center position of the reference object's key points. The total loss is calculated by adding the two difference values ​​with a weight of 1. Through continuous updates and iterations, the loss value is reduced, the model is optimized, and the output position of the reference object's key points becomes more accurate, thereby improving the accuracy of the region to be identified.

[0099] It should be noted that after determining multiple regions to be identified in this application embodiment, the test subject within the region to be identified can be directly subjected to posture recognition to determine the number of movements performed by the test subject. However, in the prior art, it is impossible to determine whether the test subject is the actual person and whether the person will be replaced during the test. Therefore, this application embodiment can perform facial recognition and tracking on the test subject to avoid situations where the test subject is replaced or the person is replaced midway, thus preventing cheating.

[0100] In some embodiments, the step of determining the region to be identified described above can be implemented by an identification model. Specifically, the model can extract features from the test site image based on at least two resolution parameters to obtain at least two target feature maps associated with the two resolution parameters; merge the at least two target feature maps to obtain a merged feature map; and perform identification processing on the reference object in the merged feature map to determine the region to be identified. The feature extraction process of this model can repeatedly exchange information in the parallel multi-resolution sub-network to improve the accuracy of feature extraction. The method of repeatedly exchanging information in the parallel multi-resolution sub-network during the model feature extraction process is the same as the method of extracting feature maps in the above embodiments, and will not be described again here for simplicity.

[0101] The model can be obtained through pre-training. The training process can be as follows: acquire multiple sample images and sample labels corresponding to each sample image in the multiple sample images, the sample labels being used to identify the location of the reference object; process each sample image in the multiple sample images based on the recognition model to be trained to obtain a first output result; determine a first difference value between the sample label and the first output result through a target loss function; train the recognition model to be trained based on the first difference value until the training termination condition is met, thus obtaining the pre-trained recognition model.

[0102] Optionally, the target loss function in the embodiments of this application may be the mean squared error loss function, the mean absolute value error loss function, the quantile loss function, etc. The embodiments of this application do not limit the target loss function of the recognition model.

[0103] Specifically, refer to Figure 4 As shown, the counting method provided in this application by way of an embodiment further includes the following steps:

[0104] Step S410: Perform facial recognition on the test subject to determine whether the test subject is a preset test object.

[0105] Facial recognition refers to the process of identifying a test subject by analyzing and comparing facial visual features, determining whether the test subject is a pre-set test subject, and preventing cheating by having someone else take the test on their behalf.

[0106] Optionally, the face recognition in this application embodiment may include any face recognition method that can perform face recognition, such as a face recognition method based on geometric features, a face recognition method based on eigenfaces, a face recognition method based on neural networks, a face recognition method based on elastic graph matching, a face recognition method based on line segment distance, or a face recognition method based on support vector machines. This application embodiment does not specifically limit the face recognition method.

[0107] In one embodiment of this application, the counting method provided in this application further includes the following steps:

[0108] Determine whether at least one edge of the test subject's face bounding box in each of the multiple regions to be identified exceeds the region boundary. If it exceeds the region boundary, issue a prompt message to indicate that the test subject may be cheating. If it does not exceed the region boundary, continue until the test time ends.

[0109] The system determines whether a test subject is cheating by determining whether at least one edge of the test subject's face bounding box in each of the multiple regions to be identified exceeds the region boundary. If at least one edge of the test subject's face bounding box exceeds the region boundary, a prompt message is issued to indicate that the test subject may be cheating. If the test subject does not exceed the region boundary, the system counts the movements made by the test subject until the end of the test time to prevent the test subject from being replaced during the test and thus cheating.

[0110] Currently, pre-trained pose classification models all use neural network models to classify the objects to be identified. First, feature extraction is required from the input sample images. In existing technologies, neural network models directly extract features from the input sample images, resulting in low-resolution target feature maps. This is not conducive to recognizing the pose of the test subject within multiple regions to be identified in each image of the target image set, and determining the number of movements performed by the test subject within each region. Therefore, electronic devices can extract features from the input samples based on at least two resolutions, avoiding the problem of poor pose classification results caused by low target feature map resolution.

[0111] Based on this, the embodiments of this application propose to extract features from each image in the target image set based on at least two resolution parameters to obtain at least two target feature maps associated with the two resolution parameters; merge the at least two target feature maps to obtain a merged feature map; and process the reference object in the merged feature map separately to determine the pose of the test subject.

[0112] Specifically, step S140 can be implemented in the following way:

[0113] Step S141: Use a pre-trained pose classification model to identify the pose of the test subject in the plurality of regions to be identified in each image of the target image set, and determine the number of movements completed by the test subject in each region to be identified; the pre-trained pose classification model is used to extract features from each image in the target image set based on at least two resolution parameters to obtain at least two target feature maps associated with the two resolution parameters; merge the at least two target feature maps to obtain a merged feature map; process the reference object in the merged feature map separately to determine the pose of the test subject.

[0114] In the embodiments provided in this application, the pre-trained pose classification model is a pre-trained image processing model. Here, the pre-trained pose classification model can be obtained by the electronic device training on sample data, or it can be obtained by the electronic device from other servers that provide models.

[0115] In this embodiment of the application, the pose of the test subject has multiple categories. Here, the pre-trained pose classification model is used to determine the pose category of the test subject in the region to be identified. Therefore, the pre-trained pose classification model is a multi-classification model. It can be understood that the pre-trained pose classification model can be a multi-classification processing model, that is, the pre-trained pose classification model can process the same image to be identified and determine the pose category of the test subject in the image to be identified.

[0116] It is understood that the pre-trained pose classification model in this application embodiment is a multi-class processing model. The electronic device performs a calculation on an image to be identified through the pre-trained pose classification model to determine the pose category of the test subject. In this way, the amount of computation in the image processing process can be reduced and the image processing speed can be improved during the pose classification process.

[0117] The training process of the pre-trained pose classification model is described in detail below. (Refer to...) Figure 5 As shown:

[0118] Step S510: Obtain multiple sample images and a sample label corresponding to each sample image in the multiple sample images; the sample label is used to identify the posture category of the test subject.

[0119] In the embodiments provided in this application, in order to identify the classification of the test subject's pose category, a model for image classification can be trained in advance. Before training the model, multiple sample data sets need to be acquired.

[0120] Here, the sample data includes multiple sample images and a sample label corresponding to each of the multiple sample images. A sample image refers to an image containing the subject's pose, enabling the trained pre-trained pose classification model to detect the subject's pose category in the image. Additionally, the sample label can include the subject's pose category label from the multiple sample images.

[0121] It should be noted that the subject's pose has more than two categories, so in the embodiments provided in this application, the pre-trained pose classification model is a multi-class processing model.

[0122] In the embodiments provided in this application, the sample labels may be manually labeled or obtained through other means. This application does not impose any limitations on these embodiments.

[0123] Step S520: Classify each of the multiple sample images based on the pose classification model to be trained to obtain a classification result; the pose classification model to be trained is used to extract features from the sample images based on at least two resolution parameters to obtain at least two target feature maps associated with the two resolution parameters; merge the at least two target feature maps to obtain a merged feature map; classify the reference object in the merged feature map to determine the pose.

[0124] In the embodiments provided in this application, deep learning technology is also required during training to build a pose classification model to be trained. This model can be a neural network model built on a backbone, or it can be built in other ways; this embodiment does not limit the specific methods used. Here, the acquired sample images can be input into the pose classification model to be trained. The model processes the sample images to obtain classification results. Specifically, the pose classification model is used to extract features from the sample images based on at least two resolution parameters, obtaining at least two target feature maps associated with the two resolution parameters; the at least two target feature maps are merged to obtain a merged feature map; and a reference object in the merged feature map is classified to determine the pose.

[0125] Understandably, the pose classification model to be trained is an initial model architecture built using deep learning techniques to detect the pose type of the test subject. For example, the pose classification model to be trained can be a neural network model based on multiple convolutional layers.

[0126] Step S530: Determine the first difference value between the sample label corresponding to each sample image and the classification result using the classification loss function.

[0127] In the embodiments provided in this application, since the pose classification model to be trained is an initial model, the classification result obtained after processing the sample data through the pose classification model is a predicted pose of the test subject, not the actual pose category of the test subject in the sample image. The sample label, however, refers to the class of the test subject's pose in the sample image. Therefore, there is a difference between the sample label and the output classification result.

[0128] Here, the difference between the sample labels and the initial output can be quantified using a classification loss function; that is, the classification loss function can be a function that estimates the degree of inconsistency between the predicted values ​​and the true values ​​of the pose classification model being trained. Generally, the greater the difference between the sample labels and the classification results, the larger the calculated initial difference value. The initial difference value can, to some extent, characterize the recognition performance of the pose classification model being trained.

[0129] Step S540: Train the pose classification model to be trained based on the first difference value until the training termination condition is met, and obtain the pre-trained pose classification model.

[0130] In the embodiments provided in this application, after determining the first difference value, the pose classification model to be trained can be iteratively trained based on the first difference value and the sample image, and finally a pre-trained pose classification model is obtained.

[0131] Specifically, the electronic device can adjust the parameters in the pose classification model to be trained based on the first difference value; that is, when training the pose classification model using the classification loss function, the electronic device performs backpropagation through the first difference value to adjust the model parameters of the pose classification model to be trained. Furthermore, the electronic device processes the sample images based on the adjusted pose classification model to obtain the second classification result.

[0132] Furthermore, the electronic device continues to determine the second difference value between the sample label and the second classification result based on the classification loss function; and, based on the second difference value, it trains the adjusted training recognition model until the training termination condition is met, and then uses the trained adjusted training posture classification model as the final trained pre-trained posture classification model.

[0133] In simple terms, an electronic device initializes the model parameters of a pose classification model to be trained. Then, it inputs sample images into the model, substituting the sample images and model parameters into the model for calculation to obtain a classification result. The purpose of training the pose classification model is to make its classification result as close as possible to the label data corresponding to the sample images. During initial training, since the model parameters are manually initialized, the classification result differs significantly from the label data. Therefore, after each classification result, the classification result and label data are substituted into a pre-defined classification loss function to calculate a difference value. This difference value is then used to update the model parameters. After iterating this process repeatedly with a large amount of sample data, a set of model parameters that makes the pose classification model's classification result very close to the label is finally obtained.

[0134] In summary, by inputting multiple sample images and the corresponding sample labels for each of the multiple sample images into the pose classification model to be trained, the classification results are obtained. The difference between the classification results and the labels corresponding to the sample images is continuously reduced through the classification loss function, so that the pose classification model reaches its optimal state and the classification results of the pose classification model are improved.

[0135] It should be noted that when using the pre-trained pose classification model to classify the pose of the test subject in this embodiment of the application, two particularly similar poses may be encountered, which are difficult to distinguish and lead to inaccurate pose classification results for the test subject. Therefore, this embodiment of the application optimizes the pose classification model to be trained by using similarity loss.

[0136] Specifically, in one embodiment of this application, before step S520 classifies the reference objects in the merged feature map and determines the pose, the following steps are also included:

[0137] From the merged feature map of the plurality of sample images, any three sample images are selected to obtain a first sample image feature map, a second sample image feature map, and a third sample image feature map; the first sample image feature map and the second sample image feature map are of the same category, and the first sample image feature map and the third sample image feature map are of different categories;

[0138] Calculate the similarity between the feature maps of the first sample image and the feature maps of the third sample image to obtain a first similarity; calculate the similarity between the feature maps of the first sample image and the feature maps of the third sample image to obtain a second similarity;

[0139] The pose classification model to be trained is optimized based on the similarity loss between the first similarity and the second similarity.

[0140] In other words, the triplet loss function can be used in this embodiment to optimize the pose classification model to be trained. Specifically, the triplet loss function obtains three sample images, each containing two categories. The first and second sample images belong to the same category, while the first and third sample images belong to different categories. By calculating the similarity between images of the same category and the similarity between images of different categories, iterative updates are made to reduce the similarity between images of the same pose category and increase the similarity between images of different pose categories.

[0141] It is understood that in the embodiments of this application, by calculating the similarity loss between the first similarity and the second similarity, the loss is iteratively updated using gradient descent. In this way, the similarity between the first sample image and the second sample image is smaller, and the similarity between the first sample image and the third sample image is larger, so that the pose classification model to be trained reaches the optimal level and improves the classification result of the pose classification model.

[0142] In the embodiments of this application, reference is made to Figure 6 As shown, step S140, determining the number of movements completed by the test subject within each area to be identified, includes the following steps:

[0143] Step S610: After identifying the posture category of the test subject using the pre-trained posture classification model, the required posture category for the test subject to complete one of the movements is set as a posture subsequence.

[0144] Step S620: Calculate the number of subsequences within a preset time period to obtain the number of movements completed by the test subject within the preset time period.

[0145] A posture subsequence is the category of postures required for a test subject to complete a movement. For example, an action may include a starting posture, an intermediate posture, and a final posture. If the starting posture is set to 0, the intermediate posture to 1, and the final posture to 2, then the test subject needs a starting posture, an intermediate posture, a final posture, an intermediate posture, and a starting posture to complete an action. That is, the posture subsequence is "01210".

[0146] Furthermore, the number of subsequences within the preset time period is calculated, that is, the number of "01210" sequences, to obtain the number of movements completed by the test subject within the preset time period.

[0147] The technical methods provided in the embodiments of this application will be described in detail below with reference to specific application scenarios.

[0148] It should be noted that the embodiments of this application can be used in different physical fitness tests, such as sit-ups or pull-ups. Taking sit-ups as an example, this application proposes a sit-up counting method based on image vision technology to prevent cheating and multi-person interference in sit-up teaching physical fitness tests. The reference object for the sit-up test site is a soft mat. The solution mainly includes the following steps:

[0149] Step 1: Area Division. In a multi-subject physical testing scenario, multiple soft mats are first placed on the ground. Using a soft mat recognition model, each location on the ground is divided into areas, forming an area list.

[0150] Step Two: Facial Recognition. During the physical fitness test preparation phase, test subjects enter the testing area. Facial recognition is performed on each subject lying on a soft mat within the area. If facial recognition is successful, the facial frame information is obtained and proceeds to Step Three. If facial recognition fails, the area list is used, and the camera zooms in to re-identify the test subjects in the failed area until a match is found in the identity feature database. After successful facial recognition, those using the footrests can enter the testing area to assist with the test; alternatively, they can use the footrests to conduct the formal test independently.

[0151] Step 3: Head Tracking. When a person lies down, the successfully recognized face bounding box represents their head information. Using a head tracking model, it is determined whether the test subject's head bounding box in each area exceeds the soft padding area. If it does, the test ends immediately, and a score is given to prevent cheating by switching test subjects. If it does not exceed the bounding box, a score is automatically given until the test time ends.

[0152] Step 4: Sit-up Counting. Using a posture classification model, identify the subject's posture within each region (supine position 0, intermediate position 1, sit-up position 2). Completing one sit-up requires the sequence "01210". Count the number of "01210" sequences in the posture sequence during the test and provide the subject's test score in real time.

[0153] refer to Figure 7 As shown, Figure 7 This is a flowchart of the sit-up counting method.

[0154] S701: Electronic equipment acquires images of the test site.

[0155] S702: The electronic device inputs the test site image into the cushion recognition model.

[0156] S703: Determine the area of ​​the pad to be identified.

[0157] Specifically, the electronic device acquires test site images by controlling a camera. The test site images contain reference objects, which are soft mats. The camera captures images of all the soft mats on the ground from an overhead view. The soft mats are placed arbitrarily, but their positions do not change after placement. Based on the actual situation, a soft mat recognition model is designed to divide the activity range of each soft mat area on the ground. The area is labeled as 1, 2, 3, ..., N (N is the total number of soft mats).

[0158] In its implementation, the cushion recognition model further includes: collecting images of cushions of the same style taken from various angles and heights, and then precisely labeling the positions of four points on the cushion within each image to create a dataset. To ensure the model's convergence direction more closely reflects real-world application scenarios, the validation set uses images of cushions taken from a top-down perspective. The cushion recognition model is based on a Convolutional Neural Network (CNN) and a multi-head self-attention (MHSA) mechanism. Its internal architecture uses high, medium, and low-resolution subnetworks in parallel to extract features and obtain rich feature representations, improving the accuracy of cushion coordinate recognition at different resolutions and scales. (Cushion recognition model architecture reference) Figure 3 As shown:

[0159] 1->2 refers to the processing between layers 1 and 2: the top layer H(k) is connected by a full-resolution CNN, and the top layer is downsampled to obtain the mid-layer M(k) connected by a mid-resolution CNN. Similarly, the bottom layer L(k) is connected by a low-resolution CNN. The transposed convolution of M(k) and L(k) in the lower layer is sampled and upsampled to input the information into the higher layer. Throughout the process, the information in the parallel multi-resolution sub-networks is repeatedly exchanged.

[0160] The process from layer 2 to layer 3 involves using the decoder part of MHSA. The feature map is divided into image blocks of equal size, multiple patch blocks are formed, and then a sequence of patch blocks is created. This patch block sequence is then quantized and positional encoding information is added. The process proceeds sequentially through normalization, MHSA, normalization again, and a Multilayer Perceptron (MLP) to obtain the global semantic information features.

[0161] In the final concat process, the top layer H(k) uses a 2->3 approach to extract features. For M(k) and L(k), transpose convolution is first performed, upsampling is applied, and then a 2->3 approach is used to obtain a high-resolution feature map of the same size, defined as f1. In this process, by integrating spatial information at different resolutions and global information at a single resolution, the feature extraction capabilities of the model at different resolutions are guaranteed.

[0162] Based on the soft mat recognition model, the locations of all soft mats in the venue are identified, and the activity range of each soft mat area is divided into areas numbered 1, 2, 3, ..., N (N is the number of soft mats).

[0163] Specifically, referring to Figure 8, the peak position of the heatmap output by the cushion recognition model is the center point of the target. However, there will also be some response values ​​around the center point. To eliminate their interference beforehand, the output of the model's heatmap is processed by max pooling with a 3x3 convolution kernel, retaining local maxima, and outputting the top k pixels of the cushion class, where top k is the number of cushion pixels output. By finding the position of the top k pixels, the deviation of the target corresponding to that point is found, and finally the target bounding box is parsed. (Reference) Figure 8a As shown,

[0164] xi=x+offset(x) (1)

[0165] yi=y+offset(y) (2)

[0166] Where xi is the x-coordinate of the center position of the output reference object keypoint, x is the x-coordinate of the center position of the actual reference object keypoint, and offset is the deviation of the x-coordinate of the center position of the reference object keypoint from the center position of the actual reference object keypoint. yi is the y-coordinate of the output reference object keypoint center position, y is the y-coordinate of the center position of the actual reference object keypoint, and offset is the deviation of the y-coordinate of the center position of the reference object keypoint from the center position of the actual reference object keypoint.

[0167] The target bounding box is calculated using formulas (1) and (2). The loss value is continuously updated and iterated to optimize the model, making the output reference object key point positions more accurate, thereby improving the accuracy of the region to be identified.

[0168] refer to Figure 8b As shown, the target boxes are finally parsed. The number of parsed target boxes is now the top k, and a threshold (0.3) is used to filter out some predictions with low confidence.

[0169] S704: Perform facial recognition on the test subject.

[0170] Specifically, the test subject enters the soft mat recognition area, and facial recognition is performed on each test subject within multiple soft mat recognition areas to determine if the test subject is a pre-set test object. If facial recognition is successful, the test subject's face bounding box information is obtained; if facial recognition fails, the area list is used, and the camera zooms in to re-perform facial recognition on the test subject in the failed area until a match is found with facial information in the identity feature database. After successful facial recognition, the person using the footrest can enter the soft mat area for assisted testing; or they can use the footrest to conduct the formal test independently. It is particularly important to note that other people can be present in the frame throughout this process without interfering with the method proposed in this invention.

[0171] S705: Tracks the face of the test subject.

[0172] Specifically, for test subjects lying down, the successfully recognized face bounding boxes are tracked using a face bounding box tracking model. The model checks if one edge of the subject's face bounding box extends beyond the padding area. If it does, a warning message is issued indicating possible cheating, the test ends immediately, and a score is given to prevent cheating by switching test subjects. If the edge does not extend beyond the padding area, the score is automatically given until the test time ends. It's worth noting that during the test counting process, face recognition is performed in conjunction with the pose classification model for the supine position (0 state). This further confirms the identity and also calibrates the tracked head bounding box position information to prevent the head model from losing tracking.

[0173] S706: Use a posture classification model to identify the posture of the test subject.

[0174] Specifically, a pre-trained pose classification model is used to identify the pose of the test subject in the multiple regions to be identified in each image of the target image set, and to determine the number of movements completed by the test subject in each region to be identified.

[0175] In the specific implementation process, the posture classification model includes the following steps: collecting images of sit-up movements in three postures from various angles and heights, taken by test subjects of different body types (male and female), and then accurately labeling the posture category of each image to create a dataset. To ensure that the model's convergence direction is closer to real-world application scenarios, the validation set uses sit-up movement images taken from a top-down angle.

[0176] The specific implementation of the pose classification model also uses CNN and MHSA. The backbone network architecture uses high, medium, and low resolution subnetworks in parallel to extract features and obtain rich feature representations, improving the accuracy of human pose classification at different resolutions and scales. (Pose classification model architecture reference) Figure 5 As shown:

[0177] 1->2: The top layer H(k) adopts a full-resolution image CNN connection. The top layer is downsampled to obtain the medium-resolution image CNN connection of the middle layer M(k). Similarly, the low-resolution CNN connection of the bottom layer L(k) can be obtained. The transposed convolution of the samples of the lower layers M(k) and L(k) is upsampled to input the information into the higher layers. Throughout the process, the information in the parallel multi-resolution sub-networks will be repeatedly exchanged.

[0178] 2->3: Utilize the decoder part of MHSA. The feature map is divided into image blocks of equal size, multiple patch blocks are formed, and then a sequence of patch blocks is created. Embedding is then applied to this patch block sequence, along with positional encoding information. This sequence is then processed sequentially through norm, multi-head attention, norm, and MLP to obtain global semantic information features.

[0179] In the concat part, the top layer H(k) uses a 2->3 approach to extract features. For M(k) and L(k), a transposed convolution is first performed, upsampling is applied, and then a 2->3 approach is used to obtain a high-resolution feature map of the same size, defined as f2. In this process, by integrating spatial information at different resolutions and global information at a single resolution, the feature extraction capabilities of the model at different resolutions are guaranteed.

[0180] In the head part, joint learning is performed by combining metric and representation methods. Based on the feature map f2 formed by the cat part, metric learning is performed using the triplet loss function. Specifically, from the merged feature map of three sample images, a first image feature map, a second sample image feature map, and a third sample image feature map are obtained. The first and second sample image feature maps belong to the same category, while the first and third sample image feature maps belong to different categories. Two poses are sampled each time, and 32 images are selected for each pose. The similarity between the first and third sample image feature maps is calculated to obtain the first similarity. The similarity between the first and third sample image feature maps is calculated to obtain the second similarity. Based on the similarity loss between the first and second similarities, the distance between images of the same pose class is reduced, and the distance between images of different pose classes is increased, thus solving the problem of small differences between the three categories.

[0181] Then, after normalization (BatchNorm, BN), f-bn is obtained. f-bn is then passed through fully connected layers (FC) to obtain feature vectors. Based on the feature vectors of the sample images, softmax is used to classify the sample images to obtain the classification results. The Cross Entropy Loss function is used for representation learning. Specifically, the one-hot encoded value of the true pose label is compared with the score output by the FC layer, and the Cross Entropy Loss is used to minimize the loss by applying the softmax value. Loss = 0.5Cross Entropy Loss + 0.5triplelet loss. It is worth noting that the purpose of using BN is to balance the various dimensions within the f2 feature map, making the subsequent Cross Entropy Loss easier to converge, eliminating the bias of the FC layer, and strengthening the compactness of the same pose class.

[0182] S707: Determine whether at least one side of the test subject's face bounding box extends beyond the area.

[0183] S708: If the test exceeds the specified range, a notification message will be sent and the test will end.

[0184] S709: If the range is not exceeded, count the number of sit-ups.

[0185] S710: Determine if the test time has been exceeded.

[0186] S711: If the test time has not been exceeded, proceed to step S707.

[0187] S712: If the test time is exceeded, the test will end and the result will be output.

[0188] It is understood that this application provides a method for preventing cheating and multi-person interference in sit-up counting based on image vision technology. It uses facial recognition and a tracking model to determine whether the test exceeds the designated mat area, thus preventing cheating. Simultaneously, by installing cameras above the testing area to capture a larger view, multiple people can be tested simultaneously with only the number of mats increased. A posture classification model is used to count sit-ups, statistically analyzing the "01210" subsequences to obtain the real-time count of each test subject's scores. Finally, experiments demonstrate that the technical solution provided by this invention can achieve automatic counting of sit-ups to prevent cheating even when multiple test subjects are taking the test and other people are present in the testing area.

[0189] For example, in sit-up application scenarios, the combination of facial recognition, region segmentation, and head tracking can prevent test subjects from cheating in multi-person scenarios.

[0190] A soft pad recognition model is constructed, which utilizes the concept of precise region division to more accurately obtain the activity range information of the test subjects, and prevents external interference while enabling multiple people to be tested simultaneously.

[0191] A posture classification model is constructed, and a new automatic sit-up counting method based on machine vision is realized by calculating the occurrence of specific subsequences in the posture sequence.

[0192] Compared to the closest existing count, the advantages of the embodiments of this application are as follows:

[0193] 1. Compared with manual methods, the embodiments of this application unify the calculation standard, which can save labor costs and solve the problems of automatic counting and subsequent manual entry of scores.

[0194] 2. Compared with methods based on hardware sensing devices, it has the function of preventing test subjects from cheating, does not require wearing any electronic devices, the test is very simple, and the test can be started by placing a soft pad, and the recognition accuracy is high.

[0195] 3. Compared with machine vision-based methods, the embodiments of this application can allow multiple people to test simultaneously, and can also prevent test subjects from cheating. It solves the problem that interference from other people in the video footage can lead to the inability to count or a sharp drop in counting accuracy.

[0196] This technical solution is applied to AI-powered sit-up testing, featuring two core functions: anti-cheating and simultaneous multi-person testing. This ensures fairness and impartiality while significantly improving testing efficiency and reducing labor costs. It also includes a feature to prevent interference from other people in the frame, making it suitable for regular classroom teaching. The application scenarios for this technical solution are also broad, including similar physical education tests such as pull-ups.

[0197] An embodiment of this application also provides an identification device, referencing... Figure 9 As shown, the identification device may include:

[0198] Acquisition unit 900 is used to acquire images of the test site;

[0199] The identification unit 910 is used to determine multiple regions to be identified based on reference objects in the test site image;

[0200] The acquisition unit 900 is further configured to acquire a target image set, the target image set including multiple images of at least one test subject moving within at least one of the multiple regions to be identified;

[0201] The classification unit 920 is also used to perform action recognition on the test subject in the plurality of regions to be identified in each image of the target image set, and to determine the number of times the test subject completes the movement in each region to be identified.

[0202] The identification unit 910 is further configured to extract features from the test site image based on at least two resolution parameters to obtain at least two target feature maps associated with the two resolution parameters; merge the at least two target feature maps to obtain a merged feature map; and perform identification processing on the reference object in the merged feature map to determine the region to be identified.

[0203] The recognition unit 910 is further configured to: sample the test site image based on the at least two resolution parameters to obtain at least two initial images; the at least two initial images correspond one-to-one with the at least two resolution parameters; sample each of the at least two initial images based on the at least two resolution parameters to obtain at least two images to be processed corresponding to each initial image; extract features from the multiple images to be processed corresponding to the at least two initial images respectively, and merge the image features of the images to be processed with the same resolution parameters to obtain at least two first feature maps; the at least two first feature maps correspond one-to-one with the at least two resolution parameters; continue to sample each of the at least two first feature maps based on the at least two resolution parameters to obtain at least two images to be processed corresponding to each first feature map, and merge and extract features from the multiple images to be processed corresponding to the at least two first images to obtain at least two second feature maps, and continue to process the second feature maps until at least two Nth feature maps are obtained; N is an integer greater than 2; and use the at least two Nth feature maps as the target feature maps.

[0204] The recognition unit 910 is further configured to divide each feature map to be processed in the plurality of images to be processed with the same resolution into a plurality of image blocks of the same size; merge the image blocks corresponding to the feature maps to be processed with the same resolution into an image block sequence; obtain at least two image block sequences with different resolutions; quantize the image block sequence and add position encoding information to obtain the at least two first feature maps.

[0205] The identification unit 910 is further configured to determine the position of the key points of the reference object based on the merged feature map, and to determine the position of the key points of the reference object as the region to be identified.

[0206] The identification unit 910 is also used to perform facial recognition on the test subject to determine whether the test subject is a preset test object.

[0207] The recognition unit 910 is further configured to determine whether at least one side of the test subject's face frame in each of the plurality of regions to be recognized exceeds the region boundary. If it exceeds the region boundary, a prompt message is issued to indicate that the test subject may be cheating. If it does not exceed the region boundary, the process continues until the test time ends.

[0208] The classification unit 920 is further configured to use a pre-trained pose classification model to identify the pose of the test subject in the plurality of regions to be identified in each image of the target image set, and to determine the number of movements completed by the test subject in each region to be identified. The pre-trained pose classification model is configured to extract features from each image in the target image set based on at least two resolution parameters to obtain at least two target feature maps associated with the two resolution parameters; merge the at least two target feature maps to obtain a merged feature map; and process the reference object in the merged feature map separately to determine the pose of the test subject.

[0209] In the embodiments provided in this application, the classification unit 920 further includes a training unit: wherein,

[0210] The training unit is further configured to acquire multiple sample images and sample labels corresponding to each of the multiple sample images; the sample labels are used to identify the pose category of the test subject; classify each of the multiple sample images based on the pose classification model to be trained to obtain a classification result; the pose classification model to be trained is configured to extract features from the sample images based on at least two resolution parameters to obtain at least two target feature maps associated with the two resolution parameters; merge the at least two target feature maps to obtain a merged feature map; classify the reference object in the merged feature map to determine the pose; determine a first difference value between the sample label corresponding to each sample image and the classification result through a classification loss function; train the pose classification model to be trained based on the first difference value until the training termination condition is met to obtain the pre-trained pose classification model.

[0211] The training unit is further configured to select any three sample images from the merged feature maps of the plurality of sample images to obtain a first sample image feature map, a second sample image feature map, and a third sample image feature map; the first sample image feature map and the second sample image feature map are of the same category, and the first sample image feature map and the third sample image feature map are of different categories; calculate the similarity between the first sample image feature map and the second sample image feature map to obtain a first similarity; calculate the similarity between the first sample image feature map and the third sample image feature map to obtain a second similarity; and optimize the pose classification model to be trained based on the similarity loss between the first similarity and the second similarity.

[0212] The classification unit 920 is further configured to identify the posture category of the test subject using the pre-trained posture classification model, and then set the posture category required for the test subject to complete one of the movements as a posture subsequence; calculate the number of the subsequences within a preset time period to obtain the number of movements completed by the test subject within the preset time period.

[0213] It is understood that the counting device provided in this application embodiment can determine the posture category of the test subject in the image by performing a single operation on the input image to be recognized, thereby reducing the amount of computation in the image processing process and improving the image processing speed.

[0214] Of course, in practical applications, such as Figure 10 As shown, the various components in this electronic device are coupled together via a bus system 102. It is understood that the bus system 102 is used to enable communication between these components. In addition to a data bus, the bus system 102 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 10 All buses are labeled as Bus System 102.

[0215] It is understood that the memory in this embodiment can be volatile memory or non-volatile memory, or both. Specifically, non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0216] The methods disclosed in the embodiments of this application can be applied to a processor or implemented by a processor. A processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above methods can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory. The processor reads information from the memory and, in conjunction with its hardware, completes the steps of the aforementioned methods.

[0217] This application also provides a computer storage medium, specifically a computer-readable storage medium. Computer instructions are stored thereon. As a first implementation, when the computer storage medium is located in an electronic device, the computer instructions, when executed by a processor, implement any step in the counting method described above in this application's embodiments.

[0218] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0219] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0220] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or at least two units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0221] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0222] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0223] It should be noted that the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0224] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A counting method, characterized in that, include: Acquire images of the test site; Feature extraction is performed on the test site image based on at least two resolution parameters to obtain at least two target feature maps associated with the two resolution parameters; The at least two target feature maps are merged to obtain a merged feature map; The reference objects in the test site image in the merged feature map are identified to determine multiple regions to be identified; Acquire a target image set, the target image set including multiple images of at least one test subject moving within at least one region to be identified in the plurality of regions to be identified; The pose of the test subject in the multiple regions to be identified in each image of the target image set is identified using a pre-trained pose classification model. After the pose category of the test subject is identified using the pre-trained pose classification model, the pose category required for the test subject to complete a movement is set as a pose subsequence. The number of subsequences within a preset time period is calculated to obtain the number of movements completed by the test subject within the preset time period. The pre-trained pose classification model is used to extract features from each image in the target image set based on at least two resolution parameters to obtain at least two target feature maps associated with the two resolution parameters. The at least two target feature maps are merged to obtain a merged feature map; The reference objects in the merged feature map are processed separately to determine the pose of the test subject.

2. The method according to claim 1, characterized in that, The step of extracting features from the test site based on at least two resolution parameters to obtain at least two target feature maps associated with the two resolution parameters includes: Based on the at least two resolution parameters, the test site image is sampled to obtain at least two initial images; the at least two initial images correspond one-to-one with the at least two resolution parameters. Based on the at least two resolution parameters, each of the at least two initial images is sampled to obtain at least two images to be processed corresponding to each initial image; Feature extraction is performed on multiple images to be processed corresponding to the at least two initial images, and the image features of images to be processed with the same resolution parameter are merged to obtain at least two first feature maps; the at least two first feature maps correspond one-to-one with the at least two resolution parameters. Based on the at least two resolution parameters, sampling processing is performed on each of the at least two first feature maps to obtain at least two images to be processed corresponding to each first feature map. Then, among the multiple images to be processed corresponding to the at least two first images, the images to be processed with the same resolution parameters are merged and feature extracted to obtain at least two second feature maps. The processing of the second feature maps continues until at least two Nth feature maps are obtained; N is an integer greater than 2. The at least two Nth feature maps are used as the target feature maps.

3. The method according to claim 2, characterized in that, The step involves extracting features from multiple images to be processed corresponding to the at least two initial images, and merging images of the images to be processed with the same resolution parameters to obtain at least two first feature maps, including: Each feature map to be processed in the plurality of images to be processed with the same resolution is divided into multiple image blocks of the same size; The image blocks corresponding to the feature maps to be processed with the same resolution are merged into an image block sequence; at least two image block sequences with different resolutions are obtained. The image block sequence is quantized and positional encoding information is added to obtain the at least two first feature maps.

4. The method according to claim 1, characterized in that, The process of identifying the merged feature map and determining the region to be identified includes: Based on the merged feature map, the location of the key points of the reference object is determined, and the location of the key points of the reference object is determined as the region to be identified.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: Facial recognition is performed on the test subject to determine whether the test subject is a preset test object.

6. The method according to any one of claims 1-4, characterized in that, The method further includes: Determine whether at least one edge of the test subject's face bounding box in each of the multiple regions to be identified exceeds the region boundary. If it exceeds the region boundary, issue a prompt message to indicate that the test subject may be cheating. If it does not exceed the region boundary, continue until the test time ends.

7. The method according to claim 1, characterized in that, The training process of the pre-trained pose classification model includes: Acquire multiple sample images and a sample label corresponding to each of the multiple sample images; the sample label is used to identify the pose category of the test subject. The pose classification model to be trained is used to classify each of the multiple sample images to obtain a classification result; the pose classification model to be trained is used to extract features from the sample images based on at least two resolution parameters to obtain at least two target feature maps associated with the two resolution parameters; the at least two target feature maps are merged to obtain a merged feature map; the reference object in the merged feature map is classified to determine the pose; The first difference value between the sample label corresponding to each sample image and the classification result is determined by using a classification loss function; The pose classification model to be trained is trained based on the first difference value until the training termination condition is met, thus obtaining the pre-trained pose classification model.

8. The method according to claim 7, characterized in that, Before classifying the reference object in the merged feature map to determine the pose, the method further includes: From the merged feature map of the plurality of sample images, any three sample images are selected to obtain a first sample image feature map, a second sample image feature map, and a third sample image feature map; the first sample image feature map and the second sample image feature map are of the same category, and the first sample image feature map and the third sample image feature map are of different categories; Calculate the similarity between the feature map of the first sample image and the feature map of the second sample image to obtain the first similarity; calculate the similarity between the feature map of the first sample image and the feature map of the third sample image to obtain the second similarity; The pose classification model to be trained is optimized based on the similarity loss between the first similarity and the second similarity.

9. A counting device, characterized in that, The device includes: The acquisition unit is used to acquire images of the test site; The identification unit is configured to extract features from the test site image based on at least two resolution parameters to obtain at least two target feature maps associated with the two resolution parameters; merge the at least two target feature maps to obtain a merged feature map; identify a reference object in the test site image in the merged feature map to determine multiple regions to be identified; the acquisition unit is further configured to acquire a target image set, the target image set including multiple images of at least one test subject moving within at least one region to be identified in the multiple regions to be identified; A classification unit is used to perform action recognition on the test subject in the plurality of regions to be identified in each image of the target image set using a pre-trained pose classification model. After identifying the pose category of the test subject using the pre-trained pose classification model, the pose category required for the test subject to complete one of the movements is set as a pose subsequence. The number of the subsequences within a preset time is calculated to obtain the number of movements completed by the test subject within the preset time. The pre-trained pose classification model is used to extract features from each image in the target image set based on at least two resolution parameters to obtain at least two target feature maps associated with the two resolution parameters. The at least two target feature maps are merged to obtain a merged feature map. The reference object in the merged feature map is processed separately to determine the pose of the test subject.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the counting method according to any one of claims 1 to 8.

11. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program that can run on the processor, the processor executing the program to implement the counting method according to any one of claims 1 to 8.

Images