Head and neck motion range measuring method and related device
Through the synchronous shooting and processing of multi-eye cameras and sliding modules, the problem of inaccurate shooting angles of fixed cameras is solved, the accuracy and flexibility of head and neck mobility measurement is improved, and automated measurement is realized.
Patent Information
- Application Number
- CN202411218395.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-30
AI Technical Summary
The existing camera-based head and neck mobility measurement methods have inaccurate shooting angles at fixed camera positions, resulting in low accuracy of measurement results.
Using multi-eye cameras and sliding modules, the head and neck images at multiple angles are obtained through the synchronous sliding of the sliding module and the synchronous shooting of the multi-eye camera, and the head and neck movements are obtained through host processing, including multiple rotation angles. range of activities.
It improves the accuracy and flexibility of head and neck mobility measurement, reduces the need for recalibration of camera parameters, and realizes automated measurements without manual operation.
Smart Images

Figure CN119014860A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of automatic control technology, and in particular to a method for measuring the range of motion of the head and neck and a related device. Background Art
[0002] Measuring the range of motion of the cervical joint is of great significance for the screening and diagnosis of cervical spondylosis. The traditional measurement methods of X-ray and CT three-dimensional reconstruction are not only costly, but also easily harmful to human health. Although non-invasive measurement equipment such as cervical range of motion meters or electric goniometers can avoid the damage to human health caused by measurement, they require manual operation by professionals and are less convenient to measure.
[0003] The camera-based head and neck range of motion measurement method has the advantages of low invasiveness and high automation, but the existing camera-based head and neck range of motion measurement methods often have the problem of inaccurate fixed camera shooting angle, which leads to low accuracy of head and neck range of motion measurement results. Therefore, how to improve the accuracy of head and neck range of motion measurement results is a technical problem that technicians in this field need to solve urgently. Summary of the invention
[0004] In view of the above problems, the present application provides a method for measuring the range of motion of the head and neck and a related device to achieve the purpose of... The specific scheme is as follows:
[0005] The first aspect of the present application provides a head and neck range of motion measurement system, comprising an acquisition device and a host computer connected to the acquisition device in communication, wherein a multi-eye camera and a sliding module are installed on the acquisition device, wherein the sliding module is composed of a plurality of sliding modules fixed in relative positions, and the multi-eye camera is composed of a plurality of cameras, and one camera is fixed on one sliding module, wherein the camera parameters of the multi-eye camera are pre-calibrated;
[0006] The sliding module is used to execute a synchronous sliding operation of multiple sliding modules in response to a sliding instruction, so that the multiple sliding modules slide to the position indicated by the sliding instruction;
[0007] The multi-camera is used to respond to the shooting instruction, perform the synchronous shooting operation of multiple cameras, obtain the head and neck images shot by each camera in the current time frame, and transmit the multiple head and neck images of the current time frame to the host;
[0008] The host is used to issue the sliding instruction to the sliding module, and when multiple sliding modules slide to the position indicated by the sliding instruction, issue the shooting instruction to the multi-eye camera; receive multiple head and neck images corresponding to each time frame, obtain the head and neck posture parameters of each time frame, and obtain the head and neck mobility based on the head and neck posture parameters of each time frame; the head and neck posture parameters include multiple rotation angles, and the head and neck mobility includes the activity range of multiple rotation angles.
[0009] In a possible implementation, the sliding module is a programmable screw linear sliding module, which includes four single modules respectively installed on the four beams of the collection booth of the frame structure, one single module is configured with a sliding module, and the sliding modules configured on the four single modules move up and down synchronously, and each camera in the multi-eye camera is respectively fixed on each sliding module through a pan-tilt head.
[0010] In a possible implementation, the host is also used to obtain a calibration verification result based on the three-dimensional coordinates of each key point of the head and neck, and the calibration verification result is used to indicate whether the multi-eye camera needs to be recalibrated; when a preset calibration timing is reached, a calibration sliding instruction is issued to the sliding module, and a calibration shooting instruction is issued to the multi-eye camera; wherein reaching the calibration timing includes reaching the calibration time of the predicted calibration cycle, and / or the calibration verification result indicates that the multi-eye camera needs to be recalibrated;
[0011] The sliding module is further used to execute a synchronous sliding operation of the plurality of sliding modules in response to the calibration sliding instruction, so that the plurality of sliding modules slide to a calibration position, wherein the calibration position is pre-configured based on a calibration plate, and a preset random texture is displayed on the calibration plate;
[0012] The multi-eye camera is also used to respond to a calibration shooting instruction, perform a synchronous shooting operation of multiple cameras, obtain a random texture image shot by each camera in a current time frame, and transmit the multiple random texture images of the current time frame to the host, so that the host can calibrate the camera parameters of the multi-eye camera based on the multiple random texture images and update the camera parameters of the multi-eye camera.
[0013] The second aspect of the present application provides a method for measuring the range of motion of the head and neck, which is applied to a collection device, wherein a multi-eye camera and a sliding module are installed on the collection device, wherein the sliding module is composed of a plurality of sliding modules with fixed relative positions, and the multi-eye camera is composed of a plurality of cameras, and one camera is fixed on one sliding module, wherein the camera parameters of the multi-eye camera are pre-calibrated; the method for measuring the range of motion of the head and neck comprises:
[0014] The sliding module performs a synchronous sliding operation of the plurality of sliding modules in response to the sliding instruction, so that the plurality of sliding modules slide to the position indicated by the sliding instruction;
[0015] The multi-eye camera responds to the shooting instruction, executes the synchronous shooting operation of multiple cameras, obtains the head and neck images taken by each camera in the current time frame, and transmits the multiple head and neck images of the current time frame to the host, so that the host; receives the multiple head and neck images corresponding to each time frame, obtains the head and neck posture parameters of each time frame, and obtains the head and neck mobility based on the head and neck posture parameters of each time frame; the head and neck posture parameters include multiple rotation angles, and the head and neck mobility includes the activity range of the multiple rotation angles.
[0016] In a possible implementation, the method for measuring the head and neck mobility further includes:
[0017] The sliding module performs a synchronous sliding operation of the plurality of sliding modules in response to the calibration sliding instruction, so that the plurality of sliding modules slide to a calibration position, wherein the calibration position is pre-configured based on a calibration plate, and a preset random texture is displayed on the calibration plate; the calibration sliding instruction is issued by the host when a preset calibration timing is reached, and reaching the calibration timing includes reaching a calibration time of a predicted calibration cycle, and / or the calibration verification result indicates that the multi-eye camera needs to be recalibrated;
[0018] The multi-eye camera executes a synchronous shooting operation of multiple cameras in response to a calibration shooting instruction, obtains a random texture image shot by each camera in a current time frame, and transmits the multiple random texture images of the current time frame to the host, so that the host calibrates the camera parameters of the multi-eye camera according to the multiple random texture images and updates the camera parameters of the multi-eye camera. The calibration shooting instruction is issued by the host after the multiple sliding modules slide to the calibration position.
[0019] The third aspect of the present application provides a method for measuring the range of motion of the head and neck, which is applied to a host, the host is communicatively connected to a collection device, a multi-eye camera and a sliding module are installed on the collection device, wherein the sliding module is composed of a plurality of sliding modules with fixed relative positions, the multi-eye camera is composed of a plurality of cameras, and one camera is fixed on one sliding module, the camera parameters of the multi-eye camera are pre-calibrated, and the method for measuring the range of motion of the head and neck comprises:
[0020] Sending a sliding instruction to the sliding module so that the plurality of sliding modules slide to the position indicated by the sliding instruction;
[0021] After the multiple sliding modules slide to the position indicated by the instruction, a shooting instruction is issued to the multi-eye camera, so that the multi-eye camera performs a synchronous shooting operation of multiple cameras, obtains the head and neck images taken by each camera in the current time frame, and transmits the multiple head and neck images of the current time frame to the host;
[0022] For each time frame, the three-dimensional coordinates of each preset key point of the head and neck are obtained according to the head and neck images corresponding to the multiple time frames;
[0023] For each time frame, the three-dimensional coordinates of each of the key points of the head and neck are input into a pre-trained activity estimation model to obtain the head and neck posture parameters output by the activity estimation model, wherein the head and neck posture parameters include multiple rotation angles, and the activity estimation model is constructed based on a neural network;
[0024] Based on the head and neck posture parameters of each time frame, the head and neck mobility is obtained, and the head and neck mobility includes a range of motion at multiple rotation angles.
[0025] In a possible implementation, issuing a sliding instruction to the sliding module so that the plurality of sliding modules slide to the position indicated by the sliding instruction includes:
[0026] Based on the height information of the person to be measured, the corresponding head and neck height to be measured is obtained; based on the head and neck height to be measured, a position sliding instruction is generated, and the position sliding instruction includes the positioning information corresponding to the head and neck height; the position sliding instruction is issued to the sliding module, so that multiple sliding modules slide to the position indicated by the positioning information; the sliding instruction includes the first sliding instruction.
[0027] In a possible implementation, issuing a sliding instruction to the sliding module so that the plurality of sliding modules slide to the position indicated by the sliding instruction includes:
[0028] Receive at least one image to be tested sent back by the multi-eye camera at a preset first frequency, identify whether the image to be tested includes a complete head and neck area, and if not, generate a second sliding instruction based on multiple preset positioning points in the image to be tested; until the image to be tested includes a complete head and neck area, wherein the second sliding instruction includes an upward sliding instruction or a downward sliding instruction, and the positioning points include multiple key points located at the edge of the head and neck; the sliding instruction includes the second sliding instruction.
[0029] In a possible implementation, the target time frame is any time frame, and the three-dimensional coordinates of each preset head and neck key point are obtained according to the head and neck image corresponding to the target time frame, including:
[0030] For each of the head and neck key points, obtain the two-dimensional coordinates of the head and neck key points in each head and neck image corresponding to the target time frame, and obtain a two-dimensional coordinate combination of the head and neck key points, wherein the two-dimensional coordinate combination of the head and neck key points includes the two-dimensional coordinates of the head and neck key points corresponding to each head and neck image;
[0031] The three-dimensional coordinates of the key points of the head and neck are obtained by calculating the combination of the two-dimensional coordinates of the key points of the head and neck through a triangulation algorithm.
[0032] In a possible implementation, in acquiring the three-dimensional coordinates of each preset head and neck key point according to the head and neck image corresponding to the target time frame, the method for measuring the head and neck activity further includes:
[0033] For each of the key points of the head and neck, based on the inverse algorithm of the triangulation algorithm, the three-dimensional coordinates of the key points of the head and neck are mapped to obtain two-dimensional coordinates corresponding to each head and neck image as the mapped two-dimensional coordinates;
[0034] Compare the two-dimensional coordinates of each of the head and neck key points mapped with the two-dimensional coordinates of the head and neck key points corresponding to each head and neck image in the combination of the two-dimensional coordinates to obtain a calibration verification result, wherein the calibration verification result is used to indicate whether the multi-eye camera needs to be recalibrated;
[0035] When a preset calibration timing is reached, a calibration sliding instruction is issued to the sliding module, wherein the calibration sliding instruction is used to control multiple sliding modules to slide to a calibration position, wherein the calibration position is pre-configured based on a calibration plate, and a preset random texture is displayed on the calibration plate;
[0036] When the plurality of sliding modules slide to the calibration position, a calibration shooting instruction is sent to the multi-eye camera, wherein the calibration shooting instruction is used to control the multi-eye camera to perform a synchronous shooting operation of the plurality of cameras, obtain a random texture image shot by each camera in the current time frame, and transmit the plurality of random texture images of the current time frame to the host;
[0037] Wherein, reaching the calibration timing includes reaching the calibration time of the predicted calibration cycle, and / or the calibration verification result indicates that recalibration is required;
[0038] The camera parameters of the multi-eye camera are calibrated according to the multiple random texture images, and the camera parameters of the multi-eye camera are updated.
[0039] The fourth aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the method for measuring the head and neck mobility of the above-mentioned first aspect or any implementation of the first aspect.
[0040] A fifth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:
[0041] The memory is used to store computer programs;
[0042] The processor is used to execute the computer program so that the electronic device can implement the method for measuring the head and neck mobility of the first aspect or any implementation of the first aspect.
[0043] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the method for measuring the head and neck mobility of the above-mentioned first aspect or any implementation of the first aspect.
[0044] By means of the above technical scheme, the present application provides a system and method for measuring the activity of the head and neck, wherein the host sends a sliding instruction to the sliding module, so that the multiple sliding modules slide to the position indicated by the sliding instruction. After the multiple sliding modules slide to the position indicated by the instruction, a shooting instruction is sent to the multi-eye camera, so that the multi-eye camera performs the synchronous shooting operation of multiple cameras, obtains the head and neck images taken by each camera in the current time frame, and transmits the multiple head and neck images of the current time frame to the host. For each time frame, the three-dimensional coordinates of each preset head and neck key point are obtained according to the head and neck images corresponding to the multiple time frames. For each time frame, the three-dimensional coordinates of each head and neck key point are input into the activity estimation model obtained in advance, and the head and neck posture parameters output by the activity estimation model are obtained. The head and neck posture parameters include multiple rotation angles, and the activity estimation model is constructed based on a neural network. Based on the head and neck posture parameters of each time frame, the head and neck activity is obtained, and the head and neck activity includes the activity range of multiple rotation angles. It can be seen that since multiple cameras are installed on multiple sliding modules that are fixed in relative positions, and the sliding modules can slide to the specified position according to the sliding instructions, the present application can achieve the acquisition of multiple head and neck images by shooting at multiple angles at the same time through a non-fixed camera position, and there is no need to recalibrate the camera parameters after the camera moves, thereby improving the positioning accuracy and flexibility of the head and neck images. Furthermore, the host calculates the head and neck mobility of the head and neck images taken at multiple angles, thereby improving the accuracy of the measurement results of the head and neck mobility. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.
[0046] Figure 1 A schematic diagram of the structure of a head and neck mobility measurement system provided in this application;
[0047] Figure 2 A schematic diagram of an optional hardware structure of a terminal provided in an embodiment of the present application;
[0048] Figure 3 A schematic diagram of an optional hardware structure of a server provided in an embodiment of the present application;
[0049] Figure 4 A schematic diagram of a method for measuring head and neck mobility provided in an embodiment of the present application;
[0050] Figure 5 A schematic diagram of a specific implementation flow of a method for measuring head and neck mobility provided in an embodiment of the present application;
[0051] Figure 6 A schematic diagram of a random texture image provided in an embodiment of the present application;
[0052] Figure 7 A schematic diagram of the distribution of key points of the head and neck provided in an embodiment of the present application;
[0053] Figure 8 A schematic diagram of head and neck mobility provided in an embodiment of the present application. DETAILED DESCRIPTION
[0054] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.
[0055] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0056] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.
[0057] In order to overcome the technical shortcomings of traditional head and neck mobility measurement methods, such as low accuracy and flexibility, an embodiment of the present application provides a head and neck mobility measurement system, which collects head and neck movement images through multiple cameras with non-fixed shooting positions but fixed relative positions, and processes the head and neck movement images through a host to obtain head and neck mobility. Since the shooting positions of the multiple cameras are not fixed, the head and neck images of different individuals can be accurately captured. Since the relative positions are fixed, the multiple cameras do not need to be recalibrated after moving, thereby improving the accuracy and flexibility of the measurement of head and neck mobility.
[0058] Reference Figure 1 , Figure 1 A schematic diagram of the structure of a head and neck mobility measurement system provided in this application, such as Figure 1 As shown, the head and neck mobility measurement system includes an acquisition device host and a host connected to the acquisition device for communication. A multi-eye camera and a sliding module are installed on the acquisition device, wherein the sliding module is composed of a plurality of sliding modules with fixed relative positions, and the multi-eye camera is composed of a plurality of cameras, and one camera is fixed on one sliding module. That is, the number of sliding modules is equal to the number of cameras of the multi-eye camera. The number of sliding modules is greater than 2, and the plurality of sliding modules are evenly arranged at different positions on the same horizontal plane, and the shooting direction of the plurality of cameras is the standing position of the person. As shown Figure 1 Taking the case where the number of cameras of the multi-eye camera is 4 as an example, the structure of a system for measuring the range of motion of the head and neck is shown.
[0059] like Figure 1As shown, the acquisition equipment is installed in the acquisition booth of rectangular frame structure. Specifically, the sliding module is a programmable screw linear sliding module, which includes single modules respectively located on four beams in the vertical direction of the acquisition booth. One single module is equipped with a sliding module (also called a slider). Each camera in the multi-eye camera is fixed on the sliding module through a pan-tilt, and moves synchronously with the synchronous up and down movement of the sliding module. The screw linear sliding module has extremely high repeatability and positioning accuracy, and the relative positions of each sliding module on the screw linear sliding module are fixed, that is, each sliding module on the screw linear sliding module moves up and down completely synchronously, so that each camera moves up and down completely synchronously.
[0060] In this embodiment, the functions of the head and neck mobility measurement system are as follows:
[0061] The sliding module is used for executing a synchronous sliding operation of a plurality of sliding modules in response to a sliding instruction, so that the plurality of sliding modules slide to the position indicated by the sliding instruction.
[0062] The multi-camera is used to respond to the shooting instruction, execute the synchronous shooting operation of multiple cameras, obtain the head and neck images shot by each camera in the current time frame, and transmit the head and neck images corresponding to the current time frame to the host. The camera parameters of each camera in the multi-camera are pre-calibrated.
[0063] The host is used to send a sliding instruction to the sliding module. When the multiple sliding modules slide to the position indicated by the sliding instruction, the host sends the shooting instruction to the multi-eye camera; receives the multi-eye vision data, and obtains the head and neck posture parameters of each time frame based on the multiple head and neck images corresponding to each time frame in the multi-eye vision data, and obtains the head and neck activity based on the head and neck posture parameters of each time frame.
[0064] In this embodiment, the head and neck posture parameters include multiple rotation angles of the head and neck. Optionally, the rotation angles specifically include: a first rotation angle, a second rotation angle and a third rotation angle, wherein the first rotation angle is the rotation angle around the x-axis direction of the spatial rectangular coordinate system, the second rotation angle is the rotation angle around the y-axis direction of the spatial rectangular coordinate system, and the third rotation angle is the rotation angle around the z-axis direction of the spatial rectangular coordinate system.
[0065] In this embodiment, the head and neck mobility includes the range of motion of multiple rotation angles of the head and neck. Optionally, the neck mobility specifically includes: a first mobility, a second mobility, and a third mobility, wherein the first mobility is the range of motion of the first rotation angle, that is, the head and neck at the first mobility is the maximum rotation angle of the head and neck around the x-axis direction of the spatial rectangular coordinate system, the second mobility is the range of motion of the second rotation angle, that is, the head and neck at the second mobility is the maximum rotation angle of the head and neck around the y-axis direction of the spatial rectangular coordinate system, the third mobility is the range of motion of the third rotation angle, and the head and neck at the third mobility is the maximum rotation angle of the head and neck around the z-axis direction of the spatial rectangular coordinate system.
[0066] It can be seen from the above technical scheme that an embodiment of the present application provides a head and neck activity measurement system, including a host and an acquisition device, on which a multi-eye camera and a sliding module are installed, wherein the sliding module is composed of a plurality of sliding modules whose relative positions are fixed, the multi-eye camera is composed of a plurality of cameras, and one camera is fixed on a sliding module, and the camera parameters of the multi-eye camera are pre-calibrated; the host is used to send control instructions (including various types of sliding instructions and shooting instructions) to the acquisition device to control the sliding module and the multi-eye camera, and the acquisition device is used to transmit multi-eye visual data to the host, specifically, the sliding module is used to respond to the sliding instruction, execute the synchronous sliding operation of the plurality of sliding modules, so that the plurality of sliding modules slide to the position indicated by the sliding instruction, the multi-eye camera is used to respond to the shooting instruction, execute the synchronous shooting operation of the plurality of cameras, obtain the head and neck images taken by each camera in the current time frame, and transmit the plurality of head and neck images of the current time frame to the host, the host is used to receive the plurality of head and neck images corresponding to each time frame, obtain the head and neck posture parameters of each time frame, and obtain the head and neck activity based on the head and neck posture parameters of each time frame. It can be seen that since multiple cameras are installed on multiple sliding modules that are fixed in relative positions, and the sliding modules can slide to the specified position according to the sliding instructions, the present application can achieve the acquisition of multiple head and neck images by shooting at multiple angles at the same time through a non-fixed camera position, and there is no need to recalibrate the camera parameters after the camera moves, thereby improving the positioning accuracy and flexibility of the head and neck images. Furthermore, the host calculates the head and neck mobility of the head and neck images taken at multiple angles, thereby improving the accuracy of the measurement results of the head and neck mobility.
[0067] exist Figure 1In a possible implementation shown, a programmable screw linear sliding module is installed on the beams at the four corners of the collection booth, and each camera can be stably installed on the slider of the sliding module through a pan / tilt. With the help of the extremely high repeatability of the screw linear sliding module (the repeatability of a single module is about 0.02mm), the accurately calibrated multi-eye camera can move up and down completely synchronously without the need for recalibration after each movement. Therefore, for different persons to be tested, the height of each camera in the multi-eye camera can be adjusted by the sliding command issued by the host without changing the relative position between the cameras, so that the head and neck of the person to be tested is located at the center of the field of view of each camera. Thus, the shooting command issued by the host controls each camera to shoot synchronously, and obtains images of the head and neck shot from multiple angles. It can be seen that collecting head and neck images through the collection booth has the advantages of non-invasiveness, high head and neck positioning accuracy, full angle, and high flexibility.
[0068] It should be noted that the head and neck mobility measurement system provided in the embodiment of the present application can be implemented through a variety of specific structures.
[0069] For example, in one possible implementation, the host is also used to obtain a calibration verification result based on the three-dimensional coordinates of each key point of the head and neck, and the calibration verification result is used to indicate whether the multi-eye camera needs to be recalibrated. When a preset calibration timing is reached, a calibration shooting instruction is issued to the multi-eye camera. After receiving the random texture image, the camera parameters of the multi-eye camera are calibrated according to multiple random texture images, and the camera parameters of the multi-eye camera are updated. Among them, reaching the calibration timing includes reaching the calibration time of the predicted calibration cycle, and / or the calibration verification result indicates that the multi-eye camera needs to be recalibrated, that is, when at least one of the calibration time of the calibration cycle is reached or the calibration verification result indicates that the multi-eye camera needs to be recalibrated, the host issues a calibration shooting instruction to the multi-eye camera.
[0070] The multi-eye camera is also used to respond to the calibration shooting instruction, perform the synchronous shooting operation of multiple cameras, obtain the random texture image taken by each camera in the current time frame, and transmit the multiple random texture images of the current time frame to the host.
[0071] It can be seen that the head and neck mobility measurement system provided in the embodiment of the present application determines whether recalibration is needed through the host, and after determining that recalibration is needed, takes multiple random texture images through a multi-eye camera, and recalibrates the camera parameters of the multi-eye camera based on the multiple random texture images through the host, thereby improving the accuracy of the camera parameters, thereby improving the accuracy of the measurement results of the head and neck mobility.
[0072] For another example, in a possible implementation, a calibration plate printed with a random texture image, or an electronic screen that can display a random texture image, is installed at a preset position in the collection booth, and the host is also used to issue a calibration sliding instruction to the sliding module, so that the sliding module performs a synchronous sliding operation of multiple sliding modules, so that the multiple sliding modules slide to the calibration position indicated by the calibration sliding instruction, so that the multi-eye camera can clearly and completely capture the random texture image, thereby improving the calibration accuracy.
[0073] For another example, there are many ways to implement the host's specific function of obtaining the range of motion of the head and neck, and the details can be found in the following embodiments.
[0074] The embodiment of the present application also provides a method for measuring the range of motion of the head and neck, the purpose of which is to measure the range of motion of the head and neck by synchronously taking multi-angle head and neck images, so as to improve the accuracy of the measurement results. The method for measuring the range of motion of the head and neck provided in the embodiment of the present application is applied to a host, the host is communicatively connected with an acquisition device, a multi-eye camera and a sliding module are installed on the acquisition device, wherein the sliding module is composed of a plurality of sliding modules with fixed relative positions, the multi-eye camera is composed of a plurality of cameras, and one camera is fixed on one sliding module, wherein the camera parameters of the multi-eye camera are pre-calibrated.
[0075] In one possible implementation, the host may be a terminal, on which a head and neck mobility measurement application may be installed. The above application and web page may provide a head and neck mobility measurement interface, and the terminal may receive relevant parameters entered by the user on the head and neck mobility measurement interface, such as collection frequency or information of the person to be measured.
[0076] Next, the product form of the terminal is described;
[0077] The terminal in the embodiment of the present application may be a mobile phone, a tablet computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiment of the present application does not impose any limitation on this.
[0078] Figure 2 An optional hardware structure diagram of the terminal 100 is shown.
[0079] refer to Figure 2As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), an earphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190 and other components. Those skilled in the art will appreciate that Figure 2 These are merely examples of terminals or multi-function devices and do not constitute limitations on the terminals or multi-function devices, which may include more or fewer components than those shown in the figures, or combinations of certain components, or different components.
[0080] The input unit 130 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the portable multifunctional device. Specifically, the input unit 130 may include a touch screen 131 (optional) and / or other input devices 132. The touch screen 131 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 170, and can receive and execute the command sent by the processor 170; the touch signal at least includes the touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.
[0081] Among them, the input device 132 can receive input data and the like.
[0082] The display unit 140 can be used to display information input by the user or information provided to the user, various menus of the terminal 100, interactive interfaces, file display and / or playback of any multimedia file. In the embodiment of the present application, the display unit 140 can be used to display the interface of the measurement of the head and neck range of motion, the processing results, etc.
[0083] The memory 120 can be used to store instructions and data. The memory 120 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc.; the instruction storage area can store software units such as operating systems, applications, instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; provide the processor 170 with hardware, software and data resources including management of computing and processing equipment, and support control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.
[0084] The processor 170 is the control center of the terminal 100. It uses various interfaces and lines to connect various parts of the entire terminal 100. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it executes various functions of the terminal 100 and processes data, thereby controlling the terminal as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 170. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on independent chips. The processor 170 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.
[0085] Among them, the memory 120 can be used to store software codes related to the method for measuring the head and neck mobility, the processor 170 can execute the steps of the method for measuring the head and neck mobility, and can also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to realize corresponding functions.
[0086] The radio frequency unit 110 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 170 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.
[0087] In this embodiment of the present application, the RF unit 110 can send data to the server 200 and receive processing results sent by the server 200.
[0088] It should be understood that the radio frequency unit 110 is optional and can be replaced by other communication interfaces, such as a network port.
[0089] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.
[0090] The terminal 100 further includes an external interface 180 , which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal 100 to communicate with other devices, or to connect a charger to charge the terminal 100 .
[0091] Although not shown, the terminal 100 may also include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which are not described in detail here. Some or all of the methods described below may be applied in the following embodiments. Figure 2 In the terminal 100 shown.
[0092] In a possible implementation, the host may be a server, and the server may be a single-site server, a cloud server, or a server cluster. Next, the product form of the server 200 is described;
[0093] Figure 3 A structural diagram of a server 200 is provided, such as Figure 3 As shown, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.
[0094] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0095] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).
[0096] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard drive (HDD), or a solid state drive (SSD).
[0097] The memory 204 may be used to store software codes related to the head and neck range of motion measurement method, and the processor 202 may execute the steps of the head and neck range of motion measurement method of the chip, and may also schedule other units to implement corresponding functions.
[0098] It should be understood that the above-mentioned terminal 100 and server 200 can be centralized or distributed devices, and the processors in the above-mentioned terminal 100 and server 200 (such as processor 170 and processor 202) can be hardware circuits (such as application specific integrated circuit (ASIC), field-programmable gate array (FPGA), general-purpose processor, digital signal processor (DSP), microprocessor or microcontroller, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.
[0099] Reference Figure 4 , Figure 4 A schematic diagram of a method for measuring head and neck mobility provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, a method for measuring the range of motion of the head and neck provided in an embodiment of the present application may include S401 to S405, and these steps are described in detail below.
[0100] S401, issuing a sliding instruction to the sliding module, so that the multiple sliding modules slide to the position indicated by the sliding instruction.
[0101] In this embodiment, the position indicated by the sliding instruction is a position such that the image captured by each camera includes the complete head and neck.
[0102] In a possible implementation, the host obtains the height of the person to be tested through the human-computer interaction interface, and obtains the corresponding head and neck height based on the height information, and generates a first sliding instruction based on the head and neck height, the first sliding instruction includes positioning information corresponding to the head and neck height, and the positioning information includes a target height. For example, if the height of the person to be tested is 1.8 meters and the head and neck height is 1.6 meters, the first sliding instruction generated by the host is used to instruct the sliding module to slide to the target height, and the target height is the height of the sliding module that makes the height of the shooting field of view of the multi-eye camera 1.6 meters. It should be noted that the correspondence between the height, the head and neck height, and the target height can be experimentally calibrated in advance.
[0103] In one possible implementation, the host receives at least one image to be tested sent back by the multi-eye camera at a preset first frequency, identifies whether the image to be tested includes a complete head and neck area, and if not, generates a sliding instruction based on a plurality of preset positioning points in the image to be tested until the image to be tested includes a complete head and neck area, wherein the second sliding instruction includes an upward sliding instruction or a downward sliding instruction. It should be noted that the positioning points include a plurality of key points located at the edge of the head and neck, and whether the image to be tested includes a complete head and neck area is determined by identifying whether the image to be tested includes all the positioning points, and the sliding instruction is generated by the number and type of the positioning points. For example, if the image to be tested lacks the positioning points located at the upper edge of the head and neck area, an upward sliding instruction is generated.
[0104] It should be noted that the head and neck area refers to an area including all key points of the head and neck, and the key points of the head and neck are key points of the human body that can indicate the movement angles of the head and neck.
[0105] S402. After the sliding instructions of the multiple sliding modules are at the positions indicated, a shooting instruction is issued to the multi-eye camera to enable the multi-eye camera to perform a synchronous shooting operation of multiple cameras, obtain the head and neck images taken by each camera in the current time frame, and transmit the multiple head and neck images of the current time frame to the host.
[0106] It should be noted that the specific methods for the host to send control instructions (including but not limited to sliding instructions and shooting instructions) to the multi-eye camera and sliding module in the acquisition device and the acquisition device to return images to the host can be referred to the prior art.
[0107] S403 . For each time frame, obtain the three-dimensional coordinates of each preset key point of the head and neck based on a plurality of head and neck images.
[0108] In this embodiment, the key points of the head and neck are pre-configured, and the key points of the head and neck include parts of the human body related to the rotation angle of the head and neck, for example, the top of the head, the center of the eyebrows, the tip of the nose, the chin, the upper edge of the left ear, the upper edge of the right ear, the lower edge of the left ear, the lower edge of the right ear, the left mandibular angle, the right mandibular angle, the left clavicle, the right clavicle, the back of the head, the spinal protrusion, the left shoulder, and the right shoulder.
[0109] S404: For each time frame, the three-dimensional coordinates of each key point of the head and neck are input into a pre-trained activity estimation model to obtain the head and neck posture parameters output by the activity estimation model.
[0110] In this embodiment, the head and neck posture parameters include multiple rotation angles. The rotation angles specifically include: a first rotation angle, a second rotation angle, and a third rotation angle, wherein the first rotation angle is a rotation angle around the x-axis direction of the spatial rectangular coordinate system, the second rotation angle is a rotation angle around the y-axis direction of the spatial rectangular coordinate system, and the third rotation angle is a rotation angle around the z-axis direction of the spatial rectangular coordinate system. The head and neck posture parameters use Euler angles to express multiple rotation angles of the head and neck, that is, the head and neck posture parameters of any time frame are recorded as , where t represents the time frame, is the first rotation angle, that is, the vertical rotation angle of the head, is the second rotation angle, that is, the horizontal rotation angle of the head, is the third rotation angle, that is, the rotation around the line of sight.
[0111] In a possible implementation, the activity estimation model is constructed based on a neural network, wherein the neural network is composed of three linear layers and activation layers between the linear layers. The activity estimation model is trained based on head and neck training data, wherein the head and neck training data includes a sample three-dimensional coordinate set labeled with head and neck posture parameters, and the sample three-dimensional coordinate set includes the three-dimensional coordinates of all key points of the head and neck.
[0112] S405: Obtain the head and neck mobility based on the head and neck posture parameters of each time frame.
[0113] In this embodiment, the range of motion of the head and neck includes a range of motion of multiple rotation angles.
[0114] In this embodiment, the head and neck mobility includes the range of motion of multiple rotation angles of the head and neck. Optionally, the neck mobility specifically includes: a first mobility, a second mobility, and a third mobility, wherein the first mobility is the range of motion of the first rotation angle, that is, the head and neck at the first mobility is the maximum rotation angle of the head and neck around the x-axis direction of the spatial rectangular coordinate system, the second mobility is the range of motion of the second rotation angle, that is, the head and neck at the second mobility is the maximum rotation angle of the head and neck around the y-axis direction of the spatial rectangular coordinate system, the third mobility is the range of motion of the third rotation angle, and the head and neck at the third mobility is the maximum rotation angle of the head and neck around the z-axis direction of the spatial rectangular coordinate system.
[0115] It can be seen from the above technical scheme that the embodiment of the present application provides a method for measuring the activity of the head and neck, in which the host sends a sliding instruction to the sliding module so that the multiple sliding modules slide to the position indicated by the sliding instruction. After the multiple sliding modules slide to the position indicated by the instruction, a shooting instruction is sent to the multi-eye camera so that the multi-eye camera performs the synchronous shooting operation of multiple cameras, obtains the head and neck images taken by each camera in the current time frame, and transmits the multiple head and neck images of the current time frame to the host. For each time frame, the three-dimensional coordinates of each preset head and neck key point are obtained according to the head and neck images corresponding to the multiple time frames. For each time frame, the three-dimensional coordinates of each head and neck key point are input into the activity estimation model obtained in advance, and the head and neck posture parameters output by the activity estimation model are obtained. The head and neck posture parameters include multiple rotation angles, and the activity estimation model is constructed based on a neural network. Based on the head and neck posture parameters of each time frame, the head and neck activity is obtained, and the head and neck activity includes the activity range of multiple rotation angles. It can be seen that since multiple cameras are installed on multiple sliding modules that are fixed in relative positions, and the sliding modules can slide to the specified position according to the sliding instructions, the present application can achieve the acquisition of multiple head and neck images by shooting at multiple angles at the same time through a non-fixed camera position, and there is no need to recalibrate the camera parameters after the camera moves, thereby improving the positioning accuracy and flexibility of the head and neck images. Furthermore, the host calculates the head and neck mobility of the head and neck images taken at multiple angles, thereby improving the accuracy of the measurement results of the head and neck mobility.
[0116] Reference Figure 5 , Figure 5 A schematic diagram of a specific implementation flow of a method for measuring head and neck mobility provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, this method specifically includes:
[0117] S501: The host receives at least one image to be tested sent back by the multi-eye camera at a preset first frequency, identifies whether the image to be tested includes a complete head and neck area, and if not, generates a second sliding instruction based on multiple preset positioning points in the image to be tested.
[0118] In this embodiment, the second sliding instruction includes an upward sliding instruction or a downward sliding instruction, and the positioning points include multiple key points located at the edge of the head and neck. For example, the positioning points include the top of the head, the chin, the left clavicle, the right clavicle, the left shoulder and the right shoulder, and the sliding instruction includes the second sliding instruction.
[0119] In this embodiment, the positioning points select multiple key points located at the edge of the head and neck. By identifying the key points at the edge of the head and neck contour, the integrity of the head and neck area can be effectively identified, that is, whether the head and neck of the person to be tested is located in the center of the field of view of each camera.
[0120] S502: The sliding module executes a synchronous sliding operation of multiple sliding modules in response to a second sliding instruction, so that the multiple sliding modules slide to the positions indicated by the sliding instruction.
[0121] S503: When the image to be tested includes the complete head and neck area, the host sends a shooting instruction to the multi-eye camera.
[0122] S504, the multi-camera performs a synchronous shooting operation of multiple cameras to obtain the head and neck images taken by each camera in the current time frame, and transmits the multiple head and neck images of the current time frame to the host.
[0123] S505. The host obtains, in each time frame, for each head and neck key point, the two-dimensional coordinates of the head and neck key point in each head and neck image corresponding to the time frame, and obtains a two-dimensional coordinate combination of the head and neck key point.
[0124] In this embodiment, the two-dimensional coordinate combination of the head and neck key points includes the two-dimensional coordinates of the head and neck key points corresponding to each head and neck image. In a possible implementation, the two-dimensional coordinates of each head and neck key point in each head and neck image are extracted using a residual network ResNet.
[0125] In this embodiment, the key points of the head and neck are distributed as follows: Figure 6 As shown, the key points of the head and neck include the top of the head, the center of the eyebrows, the tip of the nose, the chin, the upper edge of the left ear, the upper edge of the right ear, the lower edge of the left ear, the lower edge of the right ear, the left mandibular angle, the right mandibular angle, the left clavicle, the right clavicle, the back of the head, the spinal protrusion, the left shoulder, and the right shoulder.
[0126] It should be noted that the present application can describe the mobility of the human head and neck in a more detailed and accurate manner by pre-configuring dense key points of the head and neck.
[0127] S506. The host computer calculates the three-dimensional coordinates of the key points of the head and neck based on the combination of the two-dimensional coordinates of the key points of the head and neck through a triangulation algorithm.
[0128] In this embodiment, based on the camera parameters of the multi-camera and the two-dimensional coordinates of the head and neck key points corresponding to each camera, a triangulation algorithm is used to calculate the three-dimensional coordinates of each head and neck key point. The specific triangulation algorithm can be referred to the prior art.
[0129] S507. When the preset verification time is reached, the host maps the three-dimensional coordinates of each head and neck key point to obtain the two-dimensional coordinates corresponding to each head and neck image based on the inverse algorithm of the triangulation algorithm as the mapped two-dimensional coordinates.
[0130] In this embodiment, the verification timing can be pre-configured, for example, after each camera calibration, the number of sliding times of the sliding module reaches a preset sliding time threshold, the number of shots reaches a preset shooting time threshold, or the duration reaches a preset duration threshold.
[0131] S508. The host compares the two-dimensional coordinates mapped to each head and neck key point with the two-dimensional coordinates of each head and neck image in the combination of the two-dimensional coordinates of the head and neck key point to obtain a calibration verification result.
[0132] In this embodiment, the calibration verification result is used to indicate whether the multi-eye camera needs to be recalibrated.
[0133] In one possible implementation, the two-dimensional coordinates of a single head and neck key point of the same camera are compared with the mapped two-dimensional coordinates to obtain a deviation value. Based on the deviation values of multiple head and neck key points, it is determined whether the deviation exceeds a preset deviation threshold based on the deviation ratio to obtain a deviation result. Further, based on the deviation results of multiple cameras, it is determined whether recalibration is needed.
[0134] S509: When the calibration check result indicates that recalibration is required, the host sends a calibration sliding instruction to the sliding module.
[0135] S510: The sliding module executes a synchronous sliding operation of multiple sliding modules in response to the calibration sliding instruction, so that the multiple sliding modules slide to the calibration positions indicated by the calibration shooting instruction.
[0136] In this embodiment, the calibration sliding instruction is used to control multiple sliding modules to slide to the calibration position, wherein the calibration position is pre-configured based on a calibration plate, and a preset random texture is displayed on the calibration plate. For example, a calibration plate printed with a random texture image is installed at a preset position in the collection booth, or an electronic screen that can display a random texture image. The host sends a calibration sliding instruction to the sliding module, so that the sliding module performs a synchronous sliding operation of multiple sliding modules, so that the multiple sliding modules slide to the calibration position indicated by the calibration shooting instruction, so that the multi-eye camera can clearly and completely capture the random texture image, thereby improving the calibration accuracy.
[0137] In this embodiment, the random texture image displayed on the calibration board can be changed regularly. Figure 7 An optional random texture pattern is illustrated. The algorithm for generating the random texture can be found in the prior art.
[0138] S511: After the multiple sliding modules slide to the calibration positions, the host sends a calibration shooting command to the multi-eye camera.
[0139] In this embodiment, the calibration shooting instruction is used to control multiple cameras to perform synchronous shooting.
[0140] S512, the multi-camera executes a synchronous shooting operation of multiple cameras in response to the calibration shooting instruction, obtains the random texture image shot by each camera in the current time frame, and transmits the multiple random texture images of the current time frame to the host.
[0141] S513: The host calibrates the camera parameters of the multi-eye camera based on the multiple random texture images, and updates the camera parameters of the multi-eye camera.
[0142] In this embodiment, the calibration method is the same each time, that is, the camera parameters of the multi-eye camera are calibrated based on the current multiple random texture images, and the camera parameters of the multi-eye camera are updated.
[0143] It should be noted that random texture can provide a large number of feature points even when partially blocked, thereby improving the accuracy of calibration.
[0144] S514. For each time frame, the host inputs the three-dimensional coordinates of each key point of the head and neck into a pre-trained activity estimation model to obtain the head and neck posture parameters output by the activity estimation model.
[0145] In this embodiment, the head and neck posture parameters include multiple rotation angles, and the activity estimation model is constructed based on a neural network. The neural network is composed of three linear layers and activation layers between the linear layers. The activity estimation model is trained based on head and neck training data, wherein the head and neck training data includes a sample three-dimensional coordinate set with head and neck posture parameters as labels, and the sample three-dimensional coordinate set includes the three-dimensional coordinates of all key points of the head and neck.
[0146] In one possible implementation, in order to eliminate the influence of the absolute position of the human body, the three-dimensional coordinates of each key point of the head and neck can be subtracted from the three-dimensional coordinates of the spinal protrusions to obtain the standardized three-dimensional coordinates of each key point of the head and neck, and the standardized three-dimensional coordinates can be input into the neural network to obtain the head and neck posture parameters.
[0147] S515. The host obtains the head and neck mobility based on the head and neck posture parameters of each time frame.
[0148] In this embodiment, the head and neck mobility includes the range of motion of multiple rotation angles. The activity estimation model outputs the head and neck posture parameters of each time frame in the form of Euler angles in the order of time frames. In a possible implementation, a head and neck activity curve is drawn, and the head and neck activity curve includes angle change curves corresponding to each rotation angle. For each rotation angle, the maximum value and minimum value of the angle change corresponding to the rotation angle are obtained to obtain the range of motion of the rotation angle.
[0149] Figure 8 A schematic diagram of head and neck mobility provided in an embodiment of the present application, such as Figure 8 As shown, the first degree of activity includes the flexion angle and extension angle of the head and neck, and the first degree of activity is used to measure the maximum angle of the head and neck around the positive and negative directions of the x-axis of the spatial rectangular coordinate system. The second degree of activity includes the left rotation angle and the right rotation angle of the head and neck, and the second degree of activity is used to measure the maximum angle of the head and neck around the positive and negative directions of the y-axis of the spatial rectangular coordinate system. The third degree of activity includes the right flexion angle and the left flexion angle of the head and neck, and the third degree of activity is used to measure the maximum angle of the head and neck around the positive and negative directions of the z-axis of the spatial rectangular coordinate system. Among them, the x-axis is the horizontal axis along the left-right direction of the human body, the y-axis is the vertical axis, and the z-axis is the horizontal axis along the front-back direction of the human body.
[0150] It can be seen from the above technical scheme that the method for measuring the head and neck mobility provided by the embodiment of the present application, compared with the X-ray and CT three-dimensional reconstruction methods, the present invention can realize non-invasive head and neck mobility estimation. Compared with traditional cervical spine mobility measuring instruments and other equipment, the present system requires almost no manual operation when in use, and can realize automatic head and neck mobility estimation. Specifically, the present patent does not require additional equipment such as helmets when in use, which is more convenient to use, and the mobility of the person being measured will not be affected by additional equipment. The synchronous movement of multi-eye cameras is realized through the acquisition equipment, thereby reducing the system's demand for camera calibration and making it possible to collect multi-eye visual data of the person being measured in the acquisition booth without background interference.
[0151] Furthermore, the head and neck posture parameters are obtained based on the three-dimensional coordinates of the pre-configured head and neck key points. Since the head and neck key points are focused on the head and neck and have a greater impact on the head and neck posture parameters, it is possible to achieve fine reconstruction of the head and neck and improve the accuracy of the head and neck posture parameters.
[0152] Furthermore, a neural network-based activity estimation model is used to perform regression analysis on the head and neck posture parameters to improve the accuracy of the head and neck posture parameters, that is, to improve the accuracy of head and neck posture estimation.
[0153] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0154] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.
[0155] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.
Claims
1. A system for measuring head and neck mobility, characterized in that: The invention comprises a collection device and a host connected to the collection device in communication, wherein a multi-eye camera and a sliding module are installed on the collection device, wherein the sliding module is composed of a plurality of sliding modules fixed in relative positions, and the multi-eye camera is composed of a plurality of cameras, and one camera is fixed on one sliding module, wherein the camera parameters of the multi-eye camera are pre-calibrated; The sliding module is used to execute a synchronous sliding operation of multiple sliding modules in response to a sliding instruction, so that the multiple sliding modules slide to the position indicated by the sliding instruction; The multi-camera is used to respond to the shooting instruction, perform the synchronous shooting operation of multiple cameras, obtain the head and neck images shot by each camera in the current time frame, and transmit the multiple head and neck images of the current time frame to the host; The host is used to issue the sliding instruction to the sliding module, and when multiple sliding modules slide to the position indicated by the sliding instruction, issue the shooting instruction to the multi-eye camera; receive multiple head and neck images corresponding to each time frame, obtain the head and neck posture parameters of each time frame, and obtain the head and neck mobility based on the head and neck posture parameters of each time frame; the head and neck posture parameters include multiple rotation angles, and the head and neck mobility includes the activity range of multiple rotation angles.
2. The head and neck range of motion measurement system according to claim 1, characterized in that: The sliding module is a programmable screw linear sliding module, which includes four single modules respectively installed on the four beams of the collection booth with a frame structure. One single module is configured with a sliding module, and the sliding modules configured on the four single modules move up and down synchronously. Each camera in the multi-eye camera is fixed on each sliding module through a pan-tilt platform.
3. The head and neck range of motion measurement system according to claim 1, characterized in that: The host is also used to obtain a calibration verification result based on the three-dimensional coordinates of each key point of the head and neck, and the calibration verification result is used to indicate whether the multi-eye camera needs to be recalibrated; when a preset calibration timing is reached, a calibration sliding instruction is issued to the sliding module, and a calibration shooting instruction is issued to the multi-eye camera; wherein reaching the calibration timing includes reaching the calibration time of the predicted calibration cycle, and / or the calibration verification result indicates that the multi-eye camera needs to be recalibrated; The sliding module is further used to execute a synchronous sliding operation of the plurality of sliding modules in response to the calibration sliding instruction, so that the plurality of sliding modules slide to a calibration position, wherein the calibration position is pre-configured based on a calibration plate, and a preset random texture is displayed on the calibration plate; The multi-eye camera is also used to respond to a calibration shooting instruction, perform a synchronous shooting operation of multiple cameras, obtain a random texture image shot by each camera in a current time frame, and transmit the multiple random texture images of the current time frame to the host, so that the host can calibrate the camera parameters of the multi-eye camera based on the multiple random texture images and update the camera parameters of the multi-eye camera.
4. A method for measuring head and neck mobility, characterized in that: Applied to a collection device, the collection device is equipped with a multi-eye camera and a sliding module, wherein the sliding module is composed of a plurality of sliding modules fixed in relative position, the multi-eye camera is composed of a plurality of cameras, and one camera is fixed on one sliding module, wherein the camera parameters of the multi-eye camera are pre-calibrated; the method for measuring the range of motion of the head and neck includes: The sliding module performs a synchronous sliding operation of the plurality of sliding modules in response to the sliding instruction, so that the plurality of sliding modules slide to the position indicated by the sliding instruction; The multi-eye camera responds to the shooting instruction, executes the synchronous shooting operation of multiple cameras, obtains the head and neck images taken by each camera in the current time frame, and transmits the multiple head and neck images of the current time frame to the host, so that the host; receives the multiple head and neck images corresponding to each time frame, obtains the head and neck posture parameters of each time frame, and obtains the head and neck mobility based on the head and neck posture parameters of each time frame; the head and neck posture parameters include multiple rotation angles, and the head and neck mobility includes the activity range of the multiple rotation angles.
5. The method for measuring head and neck mobility according to claim 4, characterized in that: The method for measuring the range of motion of the head and neck further comprises: The sliding module performs a synchronous sliding operation of the plurality of sliding modules in response to the calibration sliding instruction, so that the plurality of sliding modules slide to a calibration position, wherein the calibration position is pre-configured based on a calibration plate, and a preset random texture is displayed on the calibration plate; the calibration sliding instruction is issued by the host when a preset calibration timing is reached, and reaching the calibration timing includes reaching a calibration time of a predicted calibration cycle, and / or the calibration verification result indicates that the multi-eye camera needs to be recalibrated; The multi-eye camera executes a synchronous shooting operation of multiple cameras in response to a calibration shooting instruction, obtains a random texture image shot by each camera in a current time frame, and transmits the multiple random texture images of the current time frame to the host, so that the host calibrates the camera parameters of the multi-eye camera according to the multiple random texture images and updates the camera parameters of the multi-eye camera. The calibration shooting instruction is issued by the host after the multiple sliding modules slide to the calibration position.
6. A method for measuring head and neck mobility, characterized in that: The method is applied to a host, the host is connected to a collection device for communication, a multi-eye camera and a sliding module are installed on the collection device, wherein the sliding module is composed of a plurality of sliding modules with fixed relative positions, the multi-eye camera is composed of a plurality of cameras, and one camera is fixed on one sliding module, the camera parameters of the multi-eye camera are pre-calibrated, and the method for measuring the range of motion of the head and neck includes: Sending a sliding instruction to the sliding module so that the plurality of sliding modules slide to the position indicated by the sliding instruction; After the multiple sliding modules slide to the position indicated by the instruction, a shooting instruction is issued to the multi-eye camera, so that the multi-eye camera performs a synchronous shooting operation of multiple cameras, obtains the head and neck images taken by each camera in the current time frame, and transmits the multiple head and neck images of the current time frame to the host; For each time frame, the three-dimensional coordinates of each preset key point of the head and neck are obtained according to the head and neck images corresponding to the multiple time frames; For each time frame, the three-dimensional coordinates of each of the key points of the head and neck are input into a pre-trained activity estimation model to obtain the head and neck posture parameters output by the activity estimation model, wherein the head and neck posture parameters include multiple rotation angles, and the activity estimation model is constructed based on a neural network; Based on the head and neck posture parameters of each time frame, the head and neck mobility is obtained, and the head and neck mobility includes a range of motion at multiple rotation angles.
7. The method for measuring head and neck mobility according to claim 6, characterized in that: The sending of a sliding instruction to the sliding module so that the plurality of sliding modules slide to the position indicated by the sliding instruction includes: Based on the height information of the person to be measured, the corresponding head and neck height to be measured is obtained; based on the head and neck height to be measured, a position sliding instruction is generated, and the position sliding instruction includes the positioning information corresponding to the head and neck height; the position sliding instruction is issued to the sliding module, so that multiple sliding modules slide to the position indicated by the positioning information; the sliding instruction includes the first sliding instruction.
8. The method for measuring head and neck mobility according to claim 6, characterized in that: The sending of a sliding instruction to the sliding module so that the plurality of sliding modules slide to the position indicated by the sliding instruction includes: Receive at least one image to be tested sent back by the multi-eye camera at a preset first frequency, identify whether the image to be tested includes a complete head and neck area, and if not, generate a second sliding instruction based on multiple preset positioning points in the image to be tested; until the image to be tested includes a complete head and neck area, wherein the second sliding instruction includes an upward sliding instruction or a downward sliding instruction, and the positioning points include multiple key points located at the edge of the head and neck; the sliding instruction includes the second sliding instruction.
9. The method for measuring head and neck mobility according to claim 6, characterized in that: The target time frame is any time frame, and the three-dimensional coordinates of each preset head and neck key point are obtained according to the head and neck image corresponding to the target time frame, including: For each of the head and neck key points, obtain the two-dimensional coordinates of the head and neck key points in each head and neck image corresponding to the target time frame, and obtain a two-dimensional coordinate combination of the head and neck key points, wherein the two-dimensional coordinate combination of the head and neck key points includes the two-dimensional coordinates of the head and neck key points corresponding to each head and neck image; The three-dimensional coordinates of the key points of the head and neck are obtained by calculating the combination of the two-dimensional coordinates of the key points of the head and neck through a triangulation algorithm.
10. The method for measuring head and neck mobility according to claim 9, characterized in that: In the step of obtaining the three-dimensional coordinates of each preset key point of the head and neck according to the head and neck image corresponding to the target time frame, the method for measuring the activity of the head and neck further includes: For each of the key points of the head and neck, based on the inverse algorithm of the triangulation algorithm, the three-dimensional coordinates of the key points of the head and neck are mapped to obtain two-dimensional coordinates corresponding to each head and neck image as the mapped two-dimensional coordinates; Compare the two-dimensional coordinates of each of the head and neck key points mapped with the two-dimensional coordinates of the head and neck key points corresponding to each head and neck image in the combination of the two-dimensional coordinates to obtain a calibration verification result, wherein the calibration verification result is used to indicate whether the multi-eye camera needs to be recalibrated; When a preset calibration timing is reached, a calibration sliding instruction is issued to the sliding module, wherein the calibration sliding instruction is used to control multiple sliding modules to slide to a calibration position, wherein the calibration position is pre-configured based on a calibration plate, and a preset random texture is displayed on the calibration plate; When the plurality of sliding modules slide to the calibration position, a calibration shooting instruction is sent to the multi-eye camera, wherein the calibration shooting instruction is used to control the multi-eye camera to perform a synchronous shooting operation of the plurality of cameras, obtain a random texture image shot by each camera in the current time frame, and transmit the plurality of random texture images of the current time frame to the host; Wherein, reaching the calibration timing includes reaching the calibration time of the predicted calibration cycle, and / or the calibration verification result indicates that recalibration is required; The camera parameters of the multi-eye camera are calibrated according to the multiple random texture images, and the camera parameters of the multi-eye camera are updated.
Citation Information
Patent Citations
Box group type non-contact three-dimensional device for automatically measuring human dimension
CN102512173A
Cervical vertebra motion degree and motion axial line position determining method, system and device
CN106447733A
Multifunctional automatic calibration system and calibration method thereof
CN107205146A
Automatic calibration method and system for multiple cameras
CN112102416A
A three-dimensional somatosensory operation recognition method and device and electronic equipment
CN113096193A
