Hunchback detection method and device, electronic equipment and computer readable storage medium

Abstract

A hump detection method, device, electronic equipment and computer readable storage medium, wherein the hump detection method comprises: acquiring an image set collected by a first camera device, wherein the first camera device comprises at least two cameras, and the image set comprises images collected by the at least two cameras simultaneously; detecting a hump key point position of a to-be-detected object and a confidence of the hump key point position in the image set by using a trained hump key point detection model; obtaining a posture feature vector of the to-be-detected object according to the hump key point position, the confidence of the hump key point position and a preset posture feature formula; and inputting the posture feature vector of the to-be-detected object into the trained hump detection model to obtain a hump detection result of the to-be-detected object. The hump detection method, device, electronic equipment and computer readable storage medium can timely and accurately detect the hump without the need for the user to wear a device.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, electronic device, and computer-readable storage medium for detecting hunchback. Background Technology

[0002] Kyphosis is a very common spinal posture abnormality. Kyphosis detection involves multiple scientific fields, including spinal surgery, sports medicine, and electromyography. Currently, the mainstream kyphosis detection methods on the market are all achieved through wearable devices controlled by external force. On the one hand, these methods require wearers to wear the device for extended periods, which is not conducive to normal bone growth and development. On the other hand, they rely on special wearable devices, which limit their applicability and usage scenarios, making it difficult to achieve the scientific goal of kyphosis correction. In other words, they cannot detect kyphosis in different scenarios in a timely manner, thus reducing the effectiveness of kyphosis detection. Summary of the Invention

[0003] This application provides a method, apparatus, electronic device, and computer-readable storage medium for detecting hunchback, which can achieve timely and accurate hunchback detection without requiring a user to wear a device.

[0004] One embodiment of this application provides a method for detecting hunchback, including:

[0005] Acquire an image set captured by a first camera device, wherein the first camera device includes at least two cameras, and the image set includes images captured simultaneously by the at least two cameras;

[0006] The trained hunchback key point detection model is used to detect the location of hunchback key points of the object to be detected in the image set and the confidence level of the location of the hunchback key points.

[0007] The posture feature vector of the object to be detected is obtained based on the location of the hunchback key point, the confidence level of the hunchback key point location, and the preset posture feature formula.

[0008] The posture feature vector of the object to be detected is input into the trained hunchback detection model to obtain the hunchback detection result of the object to be detected.

[0009] One aspect of this application also provides a hunchback detection device, including:

[0010] An image acquisition module is used to acquire an image set captured by a first camera device, wherein the first camera device includes at least two cameras, and the image set includes images captured simultaneously by the at least two cameras;

[0011] The key point detection module is used to detect the location of the hunchback key points of the object to be detected in the image set and the confidence level of the location of the hunchback key points using a trained hunchback key point detection model.

[0012] The feature extraction module is used to obtain the posture feature vector of the object to be detected based on the position of the hunchback key point, the confidence level of the position of the hunchback key point, and the preset posture feature formula.

[0013] The hunchback detection module is used to input the posture feature vector of the object to be detected into the trained hunchback detection model to obtain the hunchback detection result of the object to be detected.

[0014] This application also provides an electronic device, including: a memory and a processor; the memory stores executable program code; the processor, coupled to the memory, calls the executable program code stored in the memory to execute the hunchback detection method provided in the above embodiments.

[0015] One aspect of this application also provides a computer-readable storage medium storing a computer program thereon, which, when run by a processor, executes the hunchback detection method provided in the above embodiments.

[0016] As can be seen from the above embodiments of this application, by acquiring an image set collected by a first camera device, wherein the first camera device includes at least two cameras, and the image set includes images collected simultaneously by the at least two cameras; using a trained hunchback key point detection model to detect the hunchback key point positions and the confidence levels of the hunchback key point positions of the object to be detected in the image set; then obtaining the posture feature vector of the object to be detected based on the hunchback key point positions, the confidence levels of the hunchback key point positions, and a preset posture feature formula; and then inputting the posture feature vector of the object to be detected into the trained hunchback detection model to obtain the hunchback detection result of the object to be detected. This embodiment of the application does not require the user to wear a special device; by processing through different trained models in sequence, it can quickly and accurately identify whether hunchback exists, thereby achieving the goal of timely and accurate hunchback detection without the user wearing a device. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1A flowchart illustrating the implementation of a hunchback detection method according to an embodiment of this application;

[0019] Figure 2 This is a schematic diagram of the structure of a hunchback detection device provided in an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] See Figure 1 , Figure 1 This is a flowchart illustrating the implementation of a hunchback detection method according to an embodiment of this application. This method can be applied to electronic devices, which can be portable or fixedly installed in a predetermined location, such as... Figure 1 As shown, the method specifically includes:

[0023] Step S11: Obtain an image set captured by the first camera device, wherein the first camera device includes at least two cameras, and the image set includes images captured simultaneously by the at least two cameras;

[0024] In this embodiment, the camera device includes at least two cameras that are independent of each other or are hardware-connected. For example, the first camera device is a dual-camera device, which is a hardware device consisting of two cameras.

[0025] Specifically, when the first camera device acquires an image, at least two cameras of the first camera device acquire images simultaneously. For example, at least two cameras of the first camera device are fixed directly opposite the object to be detected and simultaneously acquire images containing the object to be detected.

[0026] In this embodiment, the image set includes one or at least two sets of images captured simultaneously by at least two cameras. For example, the first camera device includes a first camera and a second camera, and the image set includes image A1 captured by the first camera at time A, and image A2 captured by the second camera at time A. Alternatively, the first camera device includes a first camera and a second camera, and the image set includes multiple images captured by the first camera at different times within time period B, and multiple images captured by the second camera at the same times as the first camera within time period B.

[0027] Specifically, in this embodiment, at least two corresponding images in the image set (i.e., images captured by at least two cameras at the same time) include the object to be detected.

[0028] Furthermore, in an optional embodiment of the present invention, after acquiring the image set captured by the first camera device, the method further includes:

[0029] The images in the image set are subjected to noise reduction processing.

[0030] By performing noise reduction processing on the images in the image set, the recognizability of the images in the image set can be improved, thereby improving the efficiency of hunchback detection.

[0031] Step S12: Use the trained hunchback key point detection model to detect the location of the hunchback key points of the object to be detected in the image set and the confidence level of the location of the hunchback key points.

[0032] Specifically, the number of key points for hunchback can be four (such as key points for the left shoulder, right shoulder, neck, and head), which gives four key point positions for hunchback (such as key point positions for the left shoulder, right shoulder, neck, and head) and the confidence level of each key point position (such as a confidence level of 0.85 for a certain key point position for hunchback).

[0033] Furthermore, in an optional embodiment of the present invention, the step of using a trained hunchback keypoint detection model to detect the location of hunchback keypoints of the object to be detected in the image set and the confidence level of the location of the hunchback keypoints includes:

[0034] The number of objects to be detected in the image set is identified using the YOLO model;

[0035] If the number of objects to be detected in the target image of the image set is greater than a preset number, the target image is processed by image matting.

[0036] The trained hunchback key point detection model detects the location of the hunchback key points of the target object in the target image after image matting and the confidence level of the location of the hunchback key points of the target object in the target image.

[0037] In this embodiment, the YOLO model can specifically be either the YOLO2 model or the YOLO3 model. The YOLO model can quickly and simultaneously identify both the type and number of objects to be detected in an image set.

[0038] Specifically, the preset quantity can be any value greater than or equal to 2.

[0039] Image matting of a target image includes matting using a Bayesian matting algorithm.

[0040] For example, if there are more than 2 people in the image, each person in the image is processed separately to obtain an image of each person. Then, each person's image is detected by the trained hunchback keypoint detection model to obtain the hunchback keypoint position and the confidence level of the hunchback keypoint position of each person.

[0041] In this embodiment, the YOLO model and image matting processing can be used to quickly identify complex images and clearly mark the location of the hunchback key points of each object to be detected in the complex image, as well as the confidence level of the key point locations.

[0042] In this embodiment, the objects to be detected can be people of different ages and genders, such as young people, the elderly, and children. For example, the images to be detected may be frontal images of children, or side images of children.

[0043] In this embodiment, the trained hunchback key point detection model is obtained through training, and the trained hunchback key point detection model can be used to identify the hunchback key points of the object to be detected in the image.

[0044] Specifically, the location of the key points of the hunchback includes the coordinates of the key points of the hunchback.

[0045] Furthermore, in an optional embodiment of the present invention, before detecting the location of the hunchback keypoints of the object to be detected in the image and the confidence level of the location of the hunchback keypoints using the trained hunchback keypoint detection model, the method further includes:

[0046] A first sample image set is acquired by a second camera device, wherein the first camera device includes at least two second cameras, and the first sample image set includes multiple first sample images acquired simultaneously by at least two second cameras of the second camera device;

[0047] Mark the key point locations of the first object in the first sample image;

[0048] Multiple first sample images are used as input data for a preset deep neural network model, and the key point positions of the first object in the multiple first sample images are used as output data for the preset deep neural network model. The preset deep neural network model is trained to obtain the trained hunchback key point detection model.

[0049] In this embodiment, the second camera device may be the same as or different from the first camera device. Similarly, the second camera device also includes at least two cameras, which may be independent of each other or may be hardware-connected.

[0050] In this embodiment, the multiple first sample images in the first sample image set can be images of different people in different scenes and in different postures, captured simultaneously by at least two second cameras of the second camera device within multiple consecutive or non-continuous time periods.

[0051] In this embodiment, the first object is the object to be detected in the first sample image, for example, the first object is a person in the first sample. The key point location of the first object refers to the location of the key points of the first object in the first sample image. Key points are sampling points of certain parts of the object to be detected. Specifically, key points may include head key points (such as sampling points on the top of the head), left shoulder key points, right shoulder key points, or key points may also include chin key points, neck key points (such as sampling points at the junction of the neck and sternum), back key points, left shoulder key points, or right shoulder key points.

[0052] Furthermore, in an optional embodiment of the present invention, acquiring the first sample image set captured by the second camera device includes:

[0053] Acquire sample video captured by the second camera device, and perform deduplication processing on multiple frames of images in the sample video;

[0054] Type recognition is performed on the multi-frame images obtained after deduplication to obtain the type and pose of the main objects in the multi-frame images;

[0055] The first sample image set is composed of multiple frames containing subject objects of different poses and types.

[0056] In this embodiment, the number of sample videos can be one or more. Deduplication of the sample videos includes deleting images captured at intervals shorter than a preset time interval (e.g., a preset time interval of 5 seconds). Since the sample videos are captured continuously, images with the same or similar poses may appear when the time interval is short. Deleting images with the same or similar poses reduces redundancy, improves the quality of the first sample image set, and increases the efficiency of training the model.

[0057] In this embodiment, the subject can be people of different ages, such as children, youths, middle-aged people, and the elderly. The subject's posture includes standing, sitting, and walking postures.

[0058] Specifically, a preset person recognition algorithm can be used to identify the type of different frames in a sample video, or the type marking information of different frames in a sample video can be directly obtained to identify the type and posture of the main object in different frames.

[0059] Specifically, multi-frame images containing subject objects with different poses and different types refer to images where each frame contains the same or different subject objects and the same or different poses of the subject objects, or images where one or more frames contain multiple subject objects with the same or different poses.

[0060] Since the first sample image set contains different poses and different types of subject objects, the trained hunchback keypoint detection model has good generalization ability, which can improve the detection accuracy of the trained hunchback keypoint detection model.

[0061] Furthermore, in an optional embodiment of the present invention, marking the key point locations of the first object in the first sample image includes:

[0062] The camera parameters of the second camera device obtained by calibrating the second camera device are obtained. The camera parameters include the focal length F of at least two cameras, the center distance b between the at least two cameras, and the parallax d of each key point of the first object between the at least two cameras.

[0063] The depth Z of each key point is calculated according to a preset depth calculation formula, where the preset depth calculation formula Z... x =F*b / d x Where x represents the key point x, d x Z represents the parallax of keypoint x between at least two cameras. x This indicates the depth of key point x (i.e., the depth of key point x from the second camera device);

[0064] Obtain the height distance h of each key point of the first object detected from the first sample image. x The depth Z and height distance h of each key point are used as the key point positions of the first object.

[0065] Specifically, depth Z x This refers to the distance from the key point x to the depth and height h of the second camera device used to acquire the first sample image. x It refers to the height of the second camera device at which the key point x is located when acquiring the first sample image.

[0066] Specifically, calibrating the second camera device involves taking a sample image (including the object being photographed) of the second camera device, acquiring the image of the sample image, determining that there is a linear relationship between the captured sample image and the actual object being photographed in three dimensions, and calculating the focal length F of at least two cameras, the center distance b between the at least two cameras, and the parallax d of each key point between the at least two cameras based on the results of multiple image captures.

[0067] In this embodiment, the key point locations (key point locations are key point coordinates) of the first object in the first sample image set and multiple first sample images are combined as training data to train a preset deep neural network model. Specifically, during training, the first sample image set is used as the input data of the preset deep neural network model, and the key point locations of the first object in multiple first sample images are used as the output data of the preset deep neural network model, thereby performing supervised learning on the preset deep neural network model to obtain a trained hunchback key point detection model. The trained hunchback key point detection model can output key point coordinates and confidence scores for the input image.

[0068] Specifically, deep neural network models can be either convolutional neural networks (CNNs) or recurrent neural networks (RNNs). For example, a deep neural network model that is a convolutional neural network includes an input layer, convolutional layers, pooling layers, activation function layers, and fully connected layers.

[0069] Step S13: Obtain the posture feature vector of the object to be detected based on the location of the hunchback key point, the confidence level of the location of the hunchback key point, and the preset posture feature formula.

[0070] Specifically, the preset attitude feature formula can be:

[0071] α=arctan(Δz / Δh), where:

[0072] Δz=1 / 2*{(z A +z B )-(z C +z D )}

[0073] Δh=1 / 3*{h A +h B +h D}-h C

[0074] Where α is the hunchback detection angle, A, B, C, and D are key points at four different locations of the object to be detected, and z A h represents the depth of key point A from the second camera device. A The height of key point A from the second camera device; similarly, z B h represents the depth of key point B from the second camera device. B The height of key point B from the second camera device; z C Let h be the depth of the key point C from the second camera device. C The height of key point C from the second camera device; z D The depth of key point D from the second camera device, and h DThe distance from key point D to the second camera device is the height.

[0075] The pose feature vector of the object to be detected is a set of attributes of the object to be detected. For example, the pose feature vector is the attribute value of parameters such as the coordinates of each key point of the object to be detected and the hunchback detection angle.

[0076] Step S14: Input the posture feature vector of the object to be detected into the trained hunchback detection model to obtain the hunchback detection result of the object to be detected.

[0077] In this embodiment, the trained hunchback detection model can identify whether the object to be detected has a hunchback based on its posture feature vector. Specifically, the hunchback detection result is either that the object to be detected exhibits a hunchback posture, or that the object to be detected does not exhibit a hunchback posture.

[0078] Furthermore, in one embodiment of the present invention, before inputting the pose feature vector of the object to be detected into the trained hunchback detection model, the method further includes:

[0079] A first sample image set is acquired by a second camera device, wherein the first camera device includes at least two second cameras, and the first sample image set includes multiple first sample images acquired simultaneously by at least two second cameras of the second camera device;

[0080] Mark the key point locations of the first object in the first sample image;

[0081] Multiple first sample images are used as input data for a preset deep neural network model, and the key point positions of the first object in the multiple first sample images are used as output data for the preset deep neural network model. The preset deep neural network model is trained to obtain the trained hunchback key point detection model.

[0082] In this embodiment, the second sample image is a collection of images that are different from the first sample image. Specifically, the second sample image can be captured by the first camera device or the second camera device, or it can be captured by a camera device that contains at least two cameras that are different from the first camera device and the second camera device. When capturing images, at least two cameras start capturing images simultaneously.

[0083] In this embodiment, the multiple second sample images in the second sample image set can be images of different people in different scenes and in different postures collected in multiple consecutive or discontinuous time periods.

[0084] In this embodiment, the detection of the trained hunchback key point detection model and the preset posture feature formula can be consistent with the above description, which has been described in detail above and will not be repeated here.

[0085] In this embodiment, the pose feature vector of the second sample object and the hunchback annotation of the second sample image set are combined as training data to train a preset classifier model. Specifically, during training, the pose feature vector of the second sample object is used as the input data of the preset classifier model, and the hunchback annotation of the second sample image set is used as the output data of the preset classifier model, thereby performing supervised learning on the preset classifier model to obtain a trained hunchback detection model. The trained hunchback detection model can output the hunchback detection result of the object to be detected in the image based on the input image.

[0086] Furthermore, during training, the training data can be divided into data for training a pre-set classifier model and data for training the classifier model according to a preset ratio (e.g., an 8:2 or 7:3 ratio) to improve the accuracy of training the pre-set classifier model.

[0087] In this embodiment, the classifier model can be a linear classifier or a non-linear classifier. For example, the classifier model can be a Bayesian classifier or a support vector machine. Furthermore, in another embodiment of the invention, after obtaining the hunchback detection result of the object to be detected, the method further includes:

[0088] If the hunchback detection result indicates that the object to be detected has a hunchback posture, then the type of the object to be detected is obtained.

[0089] Send a reminder message or update the life plan data of the object to be detected based on the type of the object to be detected.

[0090] In this embodiment, the type of the object to be detected can be the type of the object to be detected doing different things in different environments. For example, the type of the object to be detected is the type of an adult exercising in a sports venue, or the type of a child doing homework at a table.

[0091] Specifically, sending a reminder message based on the type of the object to be detected includes sending an attitude correction reminder message to the object to be detected.

[0092] Sending reminder messages can prompt the subject to correct their posture in a timely manner, preventing prolonged hunching.

[0093] Furthermore, in another embodiment of the present invention, the step of sending a reminder message or updating the life plan data of the object to be detected according to the type of the object to be detected includes:

[0094] If the type of the object to be detected is a child, send a reminder message to the smart terminal associated with the object to be detected, or obtain the exercise plan data of the object to be detected and add a reminder plan for back exercises to the exercise plan data;

[0095] If the type of the object to be detected is an adult, send a reminder message to the smart terminal of the object to be detected, or obtain the reminder message plan data of the object to be detected and add a hunchback reminder message with a preset time interval to the reminder message plan data.

[0096] In this embodiment, the smart terminal associated with the subject of the test can be the subject's parent or family doctor. When the subject is a child, sending a reminder message to the associated smart terminal can alert the subject or the entity monitoring the subject to the subject's hunchback condition. Furthermore, it can facilitate the implementation of back exercises for the subject, thereby reducing the incidence of hunchback in the target child in the long term.

[0097] In this embodiment, when the type of the object to be detected is an adult, a reminder message is sent directly to the adult's smart terminal so that the adult can actively correct their posture. A hunchback reminder message with a preset time interval (e.g., ten or fifteen minutes) is added so that the adult can be continuously reminded so that fewer detections (e.g., one or two detections) can remind and correct the hunchback posture for a long time.

[0098] In this embodiment, an image set is acquired by a first camera device, which includes at least two cameras, and the image set includes images acquired simultaneously by the at least two cameras. A trained hunchback keypoint detection model is used to detect the hunchback keypoint positions and confidence levels of the hunchback keypoint positions of the object to be detected in the image set. Then, based on the hunchback keypoint positions, the confidence levels of the hunchback keypoint positions, and a preset posture feature formula, the posture feature vector of the object to be detected is obtained. Finally, the posture feature vector of the object to be detected is input into the trained hunchback detection model to obtain the hunchback detection result of the object to be detected. This embodiment does not require the user to wear special devices; by processing images sequentially through different trained models, it can quickly and accurately identify the presence of hunchback, thereby achieving the goal of timely and accurate hunchback detection without the need for a user-wearing device.

[0099] See Figure 2 This application provides a schematic diagram of the structure of a hunchback detection device according to an embodiment. For ease of explanation, only the parts relevant to this embodiment are shown. The device can be installed in an electronic device, which can be portable or fixedly installed in a predetermined location. The device includes:

[0100] Image acquisition module 201 is used to acquire an image set captured by a first camera device, wherein the first camera device includes at least two cameras, and the image set includes images captured simultaneously by the at least two cameras;

[0101] The key point detection module 202 is used to detect the position of the hunchback key point of the object to be detected in the image set and the confidence level of the position of the hunchback key point using a trained hunchback key point detection model.

[0102] The feature extraction module 203 is used to obtain the posture feature vector of the object to be detected based on the position of the hunchback key point, the confidence level of the position of the hunchback key point, and the preset posture feature formula.

[0103] The hunchback detection module 204 is used to input the posture feature vector of the object to be detected into the trained hunchback detection model to obtain the hunchback detection result of the object to be detected.

[0104] In this embodiment, an image set is acquired by a first camera device, which includes at least two cameras, and the image set includes images acquired simultaneously by the at least two cameras. A trained hunchback keypoint detection model is used to detect the hunchback keypoint positions and confidence levels of the hunchback keypoint positions of the object to be detected in the image set. Then, based on the hunchback keypoint positions, the confidence levels of the hunchback keypoint positions, and a preset posture feature formula, the posture feature vector of the object to be detected is obtained. Finally, the posture feature vector of the object to be detected is input into the trained hunchback detection model to obtain the hunchback detection result of the object to be detected. This embodiment does not require the user to wear special devices; by processing images sequentially through different trained models, it can quickly and accurately identify the presence of hunchback, thereby achieving the goal of timely and accurate hunchback detection without the need for a user-wearing device.

[0105] See Figure 3 The present application provides a schematic diagram of the hardware structure of an electronic device according to an embodiment.

[0106] For example, the electronic device can be any of a variety of computer system devices that are mobile or portable and perform wireless communication. Specifically, the electronic device can be a mobile phone or smartphone (e.g., an iPhone™-based phone, an Android™-based phone), a portable gaming device (e.g., a Nintendo DS™, a PlayStation Portable™, a Gameboy Advance™, an iPhone™), a laptop computer, a PDA, a portable internet device, a music player, and a data storage device, other handheld devices, and devices such as watches, headphones, pendants, etc. The electronic device can also be other wearable devices (e.g., head-mounted devices (HMDs) such as electronic glasses, electronic clothing, electronic bracelets, electronic necklaces, electronic tattoos, electronic devices, or smartwatches).

[0107] The electronic device may also be any one of a plurality of electronic devices, including but not limited to cellular phones, smartphones, other wireless communication devices, personal digital assistants, audio players, other media players, music recorders, video recorders, cameras, other media recorders, radios, medical devices, vehicle transport instruments, calculators, programmable remote controls, pagers, laptop computers, desktop computers, printers, netbooks, personal digital assistants (PDAs), portable multimedia players (PMPs), Moving Image Experts Group (MPEG-1 or MPEG-2) audio layer 3 (MP3) players, portable medical devices, and digital cameras and combinations thereof.

[0108] In some cases, electronic devices can perform multiple functions (e.g., playing music, displaying video, storing pictures, and receiving and sending telephone calls). If desired, electronic devices can be portable devices such as cellular phones, media players, other handheld devices, wristwatches, pendant devices, handset devices, or other compact portable devices.

[0109] like Figure 3 As shown, the electronic device 10 may include a control circuit, which may include a storage and processing circuit 30. The storage and processing circuit 30 may include a memory, such as a hard disk drive, a non-volatile memory (e.g., flash memory or other electronically programmable erasure-limited memory used to form a solid-state drive), a volatile memory (e.g., static or dynamic random access memory), etc., and this embodiment is not limited thereto. The processing circuit in the storage and processing circuit 30 can be used to control the operation of the electronic device 10. This processing circuit may be implemented based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio codec chips, application-specific integrated circuits (ASICs), display driver integrated circuits, etc.

[0110] The storage and processing circuitry 30 can be used to run software in the electronic device 10, such as internet browsing applications, Voice over Internet Protocol (VoIP) telephone calling applications, email applications, media playback applications, operating system functions, etc. This software can be used to perform various control operations, such as image acquisition based on a camera, ambient light measurement based on an ambient light sensor, proximity sensor measurement based on a proximity sensor, information display functions based on status indicators such as LED status lights, touch event detection based on a touch sensor, functions associated with displaying information on multiple (e.g., layered) displays, operations associated with performing wireless communication functions, operations associated with collecting and generating audio signals, control operations associated with collecting and processing button press event data, and other functions in the electronic device 10, etc., which are not limited in the embodiments of this application.

[0111] Furthermore, the memory stores executable program code, and the processor coupled to the memory calls the executable program code stored in the memory to execute the above-described process. Figure 1 The hunchback detection method described in the illustrated embodiment.

[0112] The executable program code includes the following as described above. Figure 2 The various modules in the hunchback detection device described in the illustrated embodiment include, for example, an image acquisition module, a key point detection module, a feature extraction module, and a hunchback detection module.

[0113] The electronic device 10 may further include input / output circuitry 42. Input / output circuitry 42 enables the electronic device 10 to input and output data, allowing the electronic device 10 to receive data from external devices and also allowing the electronic device 10 to output data from the electronic device 10 to external devices. Input / output circuitry 42 may further include sensors 32. Sensors 32 may include ambient light sensors, light- and capacitance-based proximity sensors, touch sensors (e.g., light-based touch sensors and / or capacitive touch sensors, wherein the touch sensor may be part of a touch display screen or used independently as a touch sensor structure), accelerometers, and other sensors, etc.

[0114] The input / output circuit 42 may also include one or more displays, such as display 14. Display 14 may include one or more of the following: liquid crystal display, organic light-emitting diode display, electronic ink display, plasma display, and displays using other display technologies. Display 14 may include a touch sensor array (i.e., display 14 may be a touch screen). The touch sensor may be a capacitive touch sensor formed by an array of transparent touch sensor electrodes (e.g., indium tin oxide (ITO) electrodes), or it may be a touch sensor formed using other touch technologies, such as acoustic touch, pressure-sensitive touch, resistive touch, optical touch, etc., which are not limited in the embodiments of this application.

[0115] The electronic device 10 may also include an audio component 36. The audio component 36 can be used to provide audio input and output functions for the electronic device 10. The audio component 36 in the electronic device 10 may include a speaker, microphone, buzzer, tone generator, and other components for generating and detecting sound.

[0116] Communication circuit 38 can be used to provide electronic device 10 with the ability to communicate with external devices. Communication circuit 38 may include analog and digital input / output interface circuits, and wireless communication circuits based on radio frequency signals and / or optical signals. The wireless communication circuit in communication circuit 38 may include radio frequency transceiver circuits, power amplifier circuits, low-noise amplifiers, switches, filters, and antennas. For example, the wireless communication circuit in communication circuit 38 may include circuitry for supporting Near Field Communication (NFC) by transmitting and receiving near-field coupled electromagnetic signals. For example, communication circuit 38 may include a near-field communication antenna and a near-field communication transceiver. Communication circuit 38 may also include cellular telephone transceivers and antennas, wireless local area network transceiver circuits and antennas, etc.

[0117] The electronic device 10 may further include a battery, power management circuitry, and other input / output units 40. Input / output units 40 may include buttons, joysticks, click wheels, scroll wheels, touchpads, keypads, keyboards, cameras, LEDs, and other status indicators.

[0118] Users can input commands through the input / output circuit 42 to control the operation of the electronic device 10, and can use the output data of the input / output circuit 42 to receive status information and other outputs from the electronic device 10.

[0119] Furthermore, embodiments of the present invention also provide a computer-readable storage medium, which may be disposed in the electronic device described in the above embodiments, and the computer-readable storage medium may be as described above. Figure 3The memory in the storage and processing circuitry 30 of the illustrated embodiment. A computer program is stored on this computer-readable storage medium, which, when executed by a processor, implements the aforementioned... Figure 1 The hunchback detection method described in the illustrated embodiment. Furthermore, the computer storage medium can also be a USB flash drive, external hard drive, read-only memory (ROM), RAM, magnetic disk, or optical disk, or any other medium capable of storing program code.

[0120] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0121] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0122] The above is a description of the hunchback detection method, apparatus, and computer-readable storage medium provided by the present invention. For those skilled in the art, based on the ideas of the embodiments of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for detecting hunchback, characterized in that, The method includes: Acquire an image set captured by a first camera device, wherein the first camera device includes at least two cameras for calculating depth information using binocular stereo vision, and the image set includes images captured simultaneously by the at least two cameras; The trained hunchback key point detection model is used to detect the location of hunchback key points of the object to be detected in the image set and the confidence level of the location of the hunchback key points. The posture feature vector of the object to be detected is obtained based on the location of the hunchback key point, the confidence level of the hunchback key point location, and the preset posture feature formula. The posture feature vector of the object to be detected is input into the trained hunchback detection model to obtain the hunchback detection result of the object to be detected; The preset posture feature formula is as follows: α=arctan(Δz / Δh), where: Δz=1 / 2*{(z A +with B )-(with C +with D )} Δh=1 / 3*{h A +h B +h D }-h C Where α is the hunchback detection angle, A, B, C, and D are key points at four different locations of the object to be detected, and z A Let h be the depth of key point A from the first camera device. A The height of key point A from the first camera device; z B h represents the depth of key point B from the first camera device. B The height of key point B from the first camera device; z C Let h be the depth of key point C from the first camera device. C The height of key point C from the first camera device; z D The key point D is the depth of the first camera device, and h D The distance from key point D to the first camera device is the height; the pose feature vector of the object to be detected includes the coordinates of each key point of the object to be detected and the attribute value of the hunchback detection angle.

2. The method according to claim 1, characterized in that, Before using the trained hunchback keypoint detection model to detect the location of hunchback keypoints of the object to be detected in the image set and the confidence level of the hunchback keypoint locations, the method further includes: A first sample image set is acquired by a second camera device, wherein the first camera device includes at least two second cameras, and the first sample image set includes multiple first sample images acquired simultaneously by at least two second cameras of the second camera device; Mark the key point locations of the first object in the first sample image; Multiple first sample images are used as input data for a preset deep neural network model, and the key point positions of the first object in the multiple first sample images are used as output data for the preset deep neural network model. The preset deep neural network model is trained to obtain the trained hunchback key point detection model.

3. The method according to claim 2, characterized in that, The acquisition of the first sample image set captured by the second camera device includes: Acquire sample video captured by the second camera device, and perform deduplication processing on multiple frames of images in the sample video; Type recognition is performed on the multi-frame images obtained after deduplication to obtain the type and pose of the main objects in the multi-frame images; The first sample image set is composed of multiple frames containing subject objects of different poses and types.

4. The method according to any one of claims 1 to 3, characterized in that, Before inputting the pose feature vector of the object to be detected into the trained hunchback detection model, the method further includes: Obtain the second set of sample images; The trained hunchback key point detection model is used to detect the location of the hunchback key point of the second sample object in the second sample image set and the confidence level of the location of the hunchback key point of the second sample object. The posture feature vector of the second sample object is obtained based on the hunchback key point location of the second sample object, the confidence level of the hunchback key point location of the second sample object, and the preset posture feature formula. The pose feature vector of the second sample object is used as the input data of a preset classifier model, and the hunchback annotation of the second sample image set is used as the output data of the preset classifier model. The preset classifier model is trained to obtain the trained hunchback detection model.

5. The method according to any one of claims 1 to 3, characterized in that, The step of using a trained hunchback keypoint detection model to detect the location of hunchback keypoints of the object to be detected in the image set and the confidence level of the location of the hunchback keypoints includes: The number of objects to be detected in the image set is identified using the YOLO model; If the number of objects to be detected in the target image of the image set is greater than a preset number, the target image is processed by image matting. The trained hunchback key point detection model detects the location of the hunchback key points of the target object in the target image after image matting and the confidence level of the location of the hunchback key points of the target object in the target image.

6. The method according to any one of claims 1 to 3, characterized in that, After obtaining the hunchback detection result of the object to be detected, the method further includes: If the hunchback detection result indicates that the object to be detected has a hunchback posture, then the type of the object to be detected is obtained. Send a reminder message or update the life plan data of the object to be detected based on the type of the object to be detected.

7. The method according to claim 6, characterized in that, The step of sending reminder messages or updating the life plan data of the object to be detected based on the type of the object to be detected includes: If the type of the object to be detected is a child, send a reminder message to the smart terminal associated with the object to be detected, or obtain the exercise plan data of the object to be detected and add a reminder plan for back exercises to the exercise plan data; If the type of the object to be detected is an adult, send a reminder message to the smart terminal of the object to be detected, or obtain the reminder message plan data of the object to be detected and add a hunchback reminder message with a preset time interval to the reminder message plan data.

8. A hunchback detection device, characterized in that, The device includes: An image acquisition module is used to acquire an image set captured by a first camera device, wherein the first camera device includes at least two cameras for calculating depth information using binocular stereo vision, and the image set includes images captured simultaneously by the at least two cameras; The key point detection module is used to detect the location of the hunchback key points of the object to be detected in the image set and the confidence level of the location of the hunchback key points using a trained hunchback key point detection model. The feature extraction module is used to obtain the posture feature vector of the object to be detected based on the position of the hunchback key point, the confidence level of the position of the hunchback key point, and the preset posture feature formula. The hunchback detection module is used to input the posture feature vector of the object to be detected into the trained hunchback detection model to obtain the hunchback detection result of the object to be detected. The preset posture feature formula is as follows: α=arctan(Δz / Δh), where: Δz=1 / 2*{(z A +with B )-(with C +with D )} Δh=1 / 3*{h A +h B +h D }-h C Where α is the hunchback detection angle, A, B, C, and D are key points at four different locations of the object to be detected, and z A Let h be the depth of key point A from the first camera device. A The height of key point A from the first camera device; z B h represents the depth of key point B from the first camera device. B The height of key point B from the first camera device; z C Let h be the depth of key point C from the first camera device. C The height of key point C from the first camera device; z D The key point D is the depth of the first camera device, and h D The distance from key point D to the first camera device is the height; the pose feature vector of the object to be detected includes the coordinates of each key point of the object to be detected and the attribute value of the hunchback detection angle.

9. An electronic device, characterized in that, The electronic device includes: Memory and processor; The memory stores executable program code; The processor coupled to the memory calls the executable program code stored in the memory to execute the hunchback detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the hunchback detection method as described in any one of claims 1 to 7.

Images