Airbag control method and device based on image recognition

Abstract

The present invention relates to the field of auxiliary medical technology, and discloses an airbag control method and device based on image recognition, the method comprising: obtaining a foot image to be analyzed; identifying a skin lesion area in the foot image to be analyzed, and determining the skin lesion characteristics of the skin lesion area; determining the skin lesion type based on the skin lesion characteristics; obtaining skin lesion monitoring data and current parameters of an airbag control system, the airbag control system comprising an airbag array, the airbag array being used to treat the foot, the parameters comprising the pressure, temperature and humidity of each airbag; determining treatment data based on the skin lesion type, the skin lesion monitoring data, and the current parameters; converting the treatment data into a control signal and transmitting it to the airbag control system, the airbag control system being used to adjust the parameters of each airbag according to the received control signal. The present invention is not only simple to implement, does not need to rely on the professional knowledge and diagnostic experience of medical personnel, but also can effectively improve the patient's treatment experience.

Description

Technical Field

[0001] The present invention relates to the field of auxiliary medical technology, and in particular to an airbag control method and device based on image recognition. Background Art

[0002] There are many common foot diseases at present, such as foot trauma, burns, ulcers, diabetic foot, foot lipoma, etc. Due to the large number of types of foot diseases, the diagnosis and treatment require high professional knowledge and diagnostic experience of medical staff, and the diagnostic process is complicated and inefficient.

[0003] In addition, some foot diseases are prone to bleeding after surgical treatment. When bleeding occurs, patients often stop bleeding by themselves or ask medical staff to stop bleeding. Common ways to stop bleeding include using elastic bandages or manual pressure. When using elastic bandages to stop bleeding, it is difficult for medical staff to accurately control the strength of the bandage. When using manual pressure to stop bleeding, the pressure and technique of each medical staff are inconsistent, which can easily increase the pain of the patient.

[0004] Therefore, there is an urgent need for a method that can quickly and accurately diagnose and treat foot diseases. Summary of the invention

[0005] In view of this, the present invention provides an airbag control method and system based on image recognition to solve the problem in the prior art that it is impossible to quickly and accurately treat the foot part that needs treatment.

[0006] In a first aspect, the present invention provides an airbag control method based on image recognition, the method comprising:

[0007] Acquire a foot image to be analyzed;

[0008] Identify the skin lesion area in the foot image to be analyzed, and determine the skin lesion characteristics of the skin lesion area;

[0009] Determine the type of lesion based on the characteristics of the lesion;

[0010] Acquire skin lesion monitoring data and current parameters of the airbag control system, the skin lesion monitoring data is obtained by detection by a sensor arranged near the skin lesion area, the airbag control system includes an airbag array, and the parameters include pressure, temperature and humidity of each airbag;

[0011] Determine treatment data based on lesion type, lesion monitoring data, and current parameters;

[0012] The treatment data is converted into a control signal and transmitted to the airbag control system, and the airbag control system is used to adjust the parameters of each airbag according to the received control signal.

[0013] The method provided in this embodiment is applicable to all foot diseases such as post-surgery, trauma, diabetic foot, etc., and can be photographed through a mobile phone, and the characteristics of the skin lesion area that needs to be pressurized or relieved are identified through image recognition, and then the pressure, temperature, humidity and other indicators near the skin lesion area are monitored, and the airbag control system is regulated in combination with the current parameters of each airbag in the airbag control system, such as pressurization, decompression, relief, temperature regulation, humidity regulation, etc., to simulate an environment suitable for skin lesion treatment or physical therapy, so as to achieve the purpose of foot treatment. This method is not only simple to implement, but also does not need to rely on the professional knowledge and diagnostic experience of medical staff, that is, it can quickly and accurately treat foot diseases, and is convenient. Moreover, this method can automatically adjust environmental parameters and pressure, which can effectively improve the patient's treatment experience, and will not cause the problem of increasing the patient's pain caused by inconsistent treatment methods of different medical staff.

[0014] In an optional implementation, identifying a skin lesion area in a foot image to be analyzed and determining skin lesion features of the skin lesion area includes:

[0015] The foot image to be analyzed is input into the skin lesion recognition model, which includes a backbone network and a branch network, wherein the backbone network constructs a high-resolution feature pyramid starting from a 1 / 4 resolution;

[0016] The foot image to be analyzed passes through the backbone network to obtain a high-resolution feature pyramid composed of feature maps of different resolutions;

[0017] The high-resolution feature pyramid is input into the branch network to obtain the skin lesion features corresponding to the skin lesion area and the coordinate values ​​corresponding to the skin lesion area.

[0018] In this embodiment, by directly building a high-resolution feature pyramid starting from 1 / 4 resolution, HigherHRNet can better retain the detail information in the input image, which is particularly important for the detection and segmentation of small-scale targets such as foot lesions. At the same time, the multi-scale fusion strategy enhances the robustness and performance of the network, allowing the entire network to maintain a high detection and segmentation accuracy in complex and changeable skin lesion situations.

[0019] In an optional implementation, the high-resolution feature pyramid is input into the branch network to obtain the skin lesion features corresponding to the skin lesion area and the coordinate values ​​corresponding to the skin lesion area, including:

[0020] Selecting the best feature map from the high-resolution feature pyramid as the feature map for detecting the skin lesion area, wherein the best feature map is the feature map that meets the preset resolution and contains the most detailed features;

[0021] Setting multiple aiming point frames of different sizes on the skin lesion area detection feature map to cover possible skin lesion areas;

[0022] Perform regression optimization on the possible skin lesion area to obtain the coordinate value corresponding to the skin lesion area;

[0023] Based on the skin lesion area detection feature map and the coordinate values ​​corresponding to the skin lesion area, the skin lesion features corresponding to the skin lesion area are extracted.

[0024] In this embodiment, the branch network can better adapt to the characteristic distribution and shape changes of the skin lesion area, thereby improving the regression accuracy of the detection frame. At the same time, since it directly receives the high-resolution feature map from HigherHRNet as input, it can make full use of the detailed information in the image to improve the accuracy of detection.

[0025] In an optional embodiment, the skin lesion recognition model is established by the following steps:

[0026] Get a sample dataset of skin lesion disease images;

[0027] Annotate the skin lesion location in each skin lesion disease image in the sample data set;

[0028] Divide the labeled sample data set into training set, validation set and test set;

[0029] Establishing an initial multi-task convolutional neural network, wherein the initial multi-task convolutional neural network includes a backbone network, a skin lesion region regression branch network, and a skin lesion region segmentation branch network, the backbone network includes a plurality of convolutional layers, pooling layers, and activation function layers of different scales connected in sequence, the skin lesion region regression branch network and the skin lesion region segmentation branch network are respectively connected to the output end of the backbone network, the skin lesion region regression branch network is used for skin lesion region positioning, and the skin lesion region segmentation branch network is used for segmenting the skin lesion region;

[0030] The labeled sample dataset is input into the initial multi-task convolutional neural network for training to obtain a skin lesion recognition model.

[0031] Both the skin lesion region regression branch network and the skin lesion region segmentation branch network can benefit from the high-resolution feature pyramid. The regression branch can more accurately predict the coordinate values ​​of the skin lesion region, while the segmentation branch can more finely divide the boundaries of the skin lesion region. The collaborative work of these two branches enables the entire network to show higher performance in skin lesion region detection and segmentation tasks.

[0032] In an optional embodiment, based on the skin lesion type, the skin lesion monitoring data, and the current parameters, the treatment data is determined, including:

[0033] Based on the type of skin lesions, determine the target pressure, target temperature and target humidity corresponding to the skin lesion area;

[0034] Based on the skin lesion monitoring data, determine the current foot pressure, foot temperature and foot humidity;

[0035] Based on the current parameters, determine the current airbag pressure, current airbag temperature and current airbag humidity of each airbag in the airbag array of the airbag control system;

[0036] Determine the pressure to be adjusted for each airbag based on the target pressure, the foot pressure, and the current airbag pressure;

[0037] Determine the temperature of each airbag to be adjusted based on the target temperature, the foot temperature, and the current airbag temperature;

[0038] Based on the target humidity, the foot humidity, and the current airbag humidity, the humidity to be adjusted for each airbag is determined.

[0039] According to the current airbag parameters of the airbag control system and the target parameters determined according to the diagnosis results, namely the target pressure, target temperature and target humidity, the parameters that need to be adjusted for the airbag, namely the current airbag pressure, current airbag temperature and current airbag humidity, can be determined. This allows accurate identification of each part of the foot that needs pressurized treatment, and accurate treatment of the part that needs treatment.

[0040] In an optional embodiment, after determining the skin lesion characteristics and skin lesion types, the method further includes:

[0041] Upload skin lesion characteristics and types to the medical knowledge base;

[0042] Based on the skin lesion characteristics, skin lesion types and medical knowledge base, the severity of the skin lesions is assessed and an assessment result is generated;

[0043] The evaluation results are sent to the user terminal for visual display.

[0044] It can help patients or doctors understand the skin lesions more clearly and further understand the progress and next steps of treatment or care.

[0045] In a second aspect, the present invention provides a foot disease treatment device based on image recognition, the device comprising:

[0046] An image acquisition module, used for acquiring a foot image to be analyzed;

[0047] An identification module, used for identifying a skin lesion area in the foot image to be analyzed and determining skin lesion features of the skin lesion area;

[0048] A determination module, used for determining the type of skin lesions based on the characteristics of the skin lesions;

[0049] A data acquisition model, used to acquire skin lesion monitoring data and current parameters of an airbag control system. The skin lesion monitoring data is detected and acquired by a sensor arranged near the skin lesion area. The airbag control system includes an airbag array, which is used to treat the foot. The parameters include pressure, temperature and humidity of each airbag.

[0050] A conversion module, used for determining treatment data based on skin lesion type, skin lesion monitoring data, and current parameters;

[0051] The communication module is used to convert the treatment data into a control signal and transmit it to the airbag control system. The airbag control system is used to adjust the parameters of each airbag according to the received control signal.

[0052] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the image recognition-based airbag control method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0053] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer instructions for enabling a computer to execute the airbag control method based on image recognition according to the first aspect or any corresponding embodiment thereof.

[0054] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the airbag control method based on image recognition according to the first aspect or any corresponding embodiment thereof.

[0055] It should be noted that, since the image recognition-based foot disease treatment device, computer device and computer-readable storage medium provided by the present invention correspond to the above-mentioned image recognition-based airbag control method. Therefore, for the beneficial effects of the image recognition-based foot disease treatment device, computer device and computer-readable storage medium, please refer to the description of the corresponding beneficial effects of the image recognition-based airbag control method above, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1is a flow chart of an airbag control method based on image recognition according to an embodiment of the present invention;

[0058] Figure 2 is a schematic diagram of positioning a skin lesion area according to an embodiment of the present invention;

[0059] Figure 3 is a structural block diagram of a foot disease treatment device based on image recognition according to an embodiment of the present invention;

[0060] Figure 4 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0062] According to an embodiment of the present invention, an embodiment of an airbag control method based on image recognition is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0063] In this embodiment, an airbag control method based on image recognition is provided, which can be executed by a terminal, a mobile terminal or other devices. Figure 1 is a flow chart of an airbag control method based on image recognition according to an embodiment of the present invention, such as Figure 1 As shown, the process includes the following steps:

[0064] Step S101, obtaining the foot image to be analyzed. Specifically, a high-resolution camera or a professional dermatoscope can be used to collect the foot image to be analyzed. These devices can capture the subtle features of the skin lesions, ensure the image quality, and ensure that the light, angle, and distance of the acquisition environment are consistent to reduce the difference in image quality. In addition, attention should be paid to the patient's comfort during acquisition to avoid additional damage to the skin. The foot image to be analyzed can be an image of the foot at any time point, such as the initial stage, treatment stage, and healing node of a foot disease.

[0065] Step S102, identifying the skin lesion area in the foot image to be analyzed, and determining the skin lesion features of the skin lesion area.

[0066] Specifically, image processing techniques such as image enhancement and denoising can be used to preprocess the collected foot images to be analyzed to improve the recognition accuracy. Then, a deep learning algorithm is used to learn features from the preprocessed foot images to be analyzed and automatically extract features. Among them, deep learning algorithms, especially convolutional neural networks, can automatically learn hierarchical feature representations from original images. These features are abstracted layer by layer from low-level edges and textures to high-level shapes and components, so that the model can process more complex images. In addition, deep learning algorithms support end-to-end learning processes, that is, the entire process from raw input to final output is completed by a unified model, without the need to manually design feature extractors and classifiers, and can automatically learn complex feature representations and use a large amount of training data for optimization. Therefore, they have better performance than traditional machine learning methods in tasks such as image recognition.

[0067] The pre-processed foot images to be analyzed are used to identify skin lesions. These deep learning algorithms can learn the characteristics of skin lesions and automatically distinguish them from normal skin. The annotated data of professional doctors can be introduced to train and optimize the algorithms to ensure the accuracy of the recognition results.

[0068] The identified skin lesion features include the size, shape, color, texture and other features of the lesions, which will be used for subsequent analysis and setting of the airbag pressure.

[0069] Specifically, regarding the size feature, the size of the lesion is one of its most intuitive features, which is usually expressed by measuring the number of pixels or the actual area of ​​the lesion area. In image recognition, edge detection algorithms such as the Canny edge detector can be used to identify the boundaries of the lesions and then calculate their areas. In addition, morphological operations such as corrosion and expansion can be used to optimize the recognition of boundaries. Traditional methods may rely on manually set thresholds or template matching to identify the size of lesions, while in this embodiment, deep learning methods are more often used to automatically identify and measure the size of lesions by training models, thereby improving accuracy and efficiency.

[0070] Regarding shape features, the shape of the lesion can reflect its pathological type or development stage. Common shape features include circular, elliptical, irregular, etc. The extraction of shape features usually involves edge detection and contour extraction of the image. After extracting the lesion contour, shape descriptors such as Hu moments and Fourier descriptors can be used to quantify its shape features. Traditional methods may rely on simple geometric shape matching to identify the shape of the lesion, while this embodiment is able to handle more complex shape changes and learn the association between the lesion shape and the disease through a deep learning model.

[0071] Regarding color features, the color of skin lesions is one of the important bases for diagnosing skin diseases. Different skin diseases may show different color features, such as red, purple, brown, etc. The extraction of color features usually involves the conversion of color space, such as conversion from RGB to HSV or Lab color space and calculation of color histogram. By comparing the color difference between the skin lesion area and the surrounding normal skin, the color features can be extracted. In this embodiment, not only the difference in color itself is paid attention to, but also more complex color features such as color distribution and color change are learned through the deep learning model, thereby improving the accuracy of diagnosis.

[0072] Regarding texture features, the texture of skin lesions refers to the visual manifestation of their surface structure, such as roughness, smoothness, granularity, etc. The extraction of texture features usually involves a variety of methods, such as gray-level co-occurrence matrix, local binary pattern, wavelet transform, etc. These methods can capture the difference in texture between the skin lesion area and the surrounding normal skin. Traditional methods may rely on manually designed texture feature extractors, while this embodiment uses more deep learning methods to automatically learn the texture features of skin lesions through models such as convolutional neural networks, thereby improving the robustness and generalization ability of the features.

[0073] In this embodiment, a feature extraction method is adopted, and a high degree of automation of skin lesion feature extraction is achieved through the combination of deep learning and image processing technology. Compared with the traditional method that requires manual intervention and parameter setting, the present embodiment can automatically identify and extract skin lesion features, greatly improving efficiency and accuracy. In this embodiment, not only the basic features of skin lesions, such as size and shape, are focused on, but also more complex features, such as color distribution, texture changes, etc., can be learned, so as to more comprehensively describe the pathological characteristics of skin lesions. Through large-scale data training and optimization of deep learning models, in this embodiment, high recognition accuracy and generalization ability can be maintained under different populations, different skin types, and different lighting conditions. With the improvement of computing power and optimization of algorithms, the extraction and identification of skin lesion features can be completed in a shorter time in this embodiment, providing timely support for clinical diagnosis and treatment. The severity of skin lesions can be evaluated in combination with the medical knowledge base in the later stage.

[0074] Step S103, based on the skin lesion characteristics, determine the skin lesion type. Specifically, the skin lesion type can be determined based on the identified skin lesion characteristics, such as size, shape, color, texture, etc. For example, for tinea pedis, the skin lesion characteristics are: dry skin, desquamation, sometimes accompanied by blisters, cracks or ulcers, etc.; for diabetic foot, the skin lesion characteristics are: redness, swelling, foot deformity, such as hammertoe, dryness, roughness, etc. It can be used to identify and determine all foot diseases and is applicable to all patients with foot diseases, and will not be listed one by one here.

[0075] Step S104, obtaining skin lesion monitoring data and current parameters of the airbag control system. The skin lesion monitoring data is obtained by detection by a sensor arranged near the skin lesion area. The airbag control system includes an airbag array, which is used to treat the foot. The parameters include the pressure, temperature and humidity of each airbag.

[0076] Specifically, sensors, such as temperature sensors, pressure sensors, etc., can be installed at or near the lesion site to monitor the physiological indicators of the lesion in real time, such as temperature, humidity, pressure, etc. Among them, temperature sensors suitable for the skin surface can be selected, such as thermistors, infrared temperature sensors, etc. These sensors should be able to accurately measure the temperature of the lesion area without irritating or damaging the skin. Sensors that can measure the humidity on the skin surface can be selected to ensure that they can maintain stable measurement accuracy under different environmental conditions. The temperature sensor and humidity sensor are tightly attached to the healthy skin near the lesion area to reduce measurement errors. At the same time, ensure good contact between the sensor and the skin to avoid air gaps affecting the measurement results.

[0077] In addition, it is necessary to ensure that the sensor data and the skin lesion image are synchronized in acquisition time, which can be achieved by setting a unified timestamp in the data acquisition system.

[0078] During real-time monitoring, the privacy of the patient should be protected. Avoid leaking sensitive information to unauthorized personnel or institutions. It is also necessary to calibrate the sensors and image acquisition equipment regularly to ensure the accuracy and reliability of the measurement results. In addition, in this embodiment, a complete data security mechanism is also established to ensure that the collected data is not tampered with or leaked during transmission, storage and processing.

[0079] Regarding the airbag control system, it can be a tension-reducing shoe for quantitative pressurization and hemostasis, the tension-reducing shoe includes an upper sole, a shoe body placed on the upper sole, and a lower sole detachably connected to the bottom of the upper sole; the shoe body includes a front part of the shoe body for wrapping the forefoot and toes of the foot, a middle part of the shoe body for wrapping the midfoot surface of the foot, and a rear part of the shoe body for wrapping the rearfoot surface and heel of the foot. Of course, the shoe body can be divided into multiple partitions, which are not limited to the above three partitions in this embodiment and can be set according to actual needs; the size of each part of the shoe body is adjustable, and each part of the shoe body includes an airbag that can be quantitatively pressurized, that is, an airbag array. In addition, in this embodiment, each airbag is also equipped with a temperature controller, a humidity controller and a control unit, and the control unit is respectively connected to the temperature controller and the humidity controller, and is used to control the temperature controller and the humidity controller of each airbag according to the control signal to achieve temperature regulation and humidity regulation. When used specifically, the patient can wear the tension-reducing shoe for treatment.

[0080] Step S105, determining treatment data based on the skin lesion type, skin lesion monitoring data, and current parameters.

[0081] In this embodiment, the lesion type corresponds to the lesion feature, and the lesion feature is fused and analyzed with the lesion monitoring data, which can be achieved by establishing a mathematical model or using a machine learning algorithm. For example, a deep learning model can be trained to take the foot lesion image features and foot physiological indicators as input, and output the diagnosis results or disease assessment of the foot disease. Then the sensor data is fused and analyzed with the lesion type of the lesion image to determine the parameters that need to be adjusted in the airbag control system, such as pressure, temperature, humidity, etc. Among them, adjusting the pressure applied to the foot can provide sufficient support and comfort, which is conducive to the healing of foot diseases; adjusting the temperature and humidity of the foot can reduce the risk of infection and improve the healing efficiency of foot diseases.

[0082] Step S106, converting the treatment data into a control signal and transmitting it to the airbag control system, and the airbag control system is used to adjust the parameters of each airbag according to the received control signal.

[0083] Specifically, the treatment data can be converted into a control signal adapted to the airbag control system through a controller or a microprocessor, and the control signal can be transmitted to the airbag control system wirelessly or wiredly. The airbag inflation volume, temperature, pressure and other parameters can be adjusted according to the control signal to simulate an environment suitable for skin lesion treatment or physical therapy, etc., thereby achieving the purpose of foot treatment.

[0084] Pressure monitoring, temperature and humidity, etc. can be performed through the mobile phone, and adjustments can be made through the mobile phone. The foot pressure and environment can be accurately and dynamically adjusted by controlling the airbag control system. The partitioned parts can be pressurized by countless airbags, and the temperature and humidity monitoring is completed by pressure sensors and temperature and humidity sensors. It can be accurate to each part of the foot that needs pressurized treatment, and give accurate treatment to the parts that need treatment. Of course, in addition to using mobile phones, servers, terminals, etc. to implement the airbag control method provided in this embodiment, it can also be applied to other wearable devices, such as: smart watches, smart insoles, and other devices that can process data.

[0085] The method provided in this embodiment is applicable to all foot diseases such as post-surgery, trauma, diabetic foot, etc., and can be photographed through a mobile phone, and the characteristics of the skin lesion area that needs to be pressurized or relieved are identified through image recognition, and then the pressure, temperature, humidity and other indicators near the skin lesion area are monitored, and the airbag control system is regulated in combination with the current parameters of each airbag in the airbag control system, such as pressurization, decompression, relief, temperature regulation, humidity regulation, etc., to simulate an environment suitable for skin lesion treatment or physical therapy, so as to achieve the purpose of foot treatment. This method is not only simple to implement, but also does not need to rely on the professional knowledge and diagnostic experience of medical staff, that is, it can quickly and accurately treat foot diseases, and is convenient. Moreover, this method can automatically adjust environmental parameters and pressure, which can effectively improve the patient's treatment experience, and will not cause the problem of increasing the patient's pain caused by inconsistent treatment methods of different medical staff.

[0086] In some optional implementations, identifying a skin lesion area in a foot image to be analyzed and determining skin lesion features of the skin lesion area includes:

[0087] The foot image to be analyzed is input into the skin lesion recognition model, which includes a backbone network and a branch network. The backbone network constructs a high-resolution feature pyramid starting from a 1 / 4 resolution.

[0088] The foot image to be analyzed passes through the backbone network to obtain a high-resolution feature pyramid composed of feature maps of different resolutions.

[0089] The high-resolution feature pyramid is input into the branch network to obtain the skin lesion features corresponding to the skin lesion area and the coordinate values ​​corresponding to the skin lesion area.

[0090] Regarding the skin lesion recognition model, a multi-task convolutional neural network is used in this embodiment. The network consists of a CNN backbone network and two branch networks, one branch for skin lesion region regression and the other branch for skin lesion region segmentation. This network structure can effectively handle the two tasks of skin lesion region positioning and skin lesion region segmentation at the same time. Specifically, one of the branch networks regresses the coordinate value of the skin lesion region detection frame, and the other branch network outputs the skin lesion region segmentation result.

[0091] The CNN backbone network uses a backbone network (HigherHRNet), which is a multi-scale high-resolution network that generates high-resolution heat maps through a new high-resolution feature pyramid module. Different from the traditional feature pyramid that starts from 1 / 32 resolution and gradually increases the feature map resolution to 1 / 4 using bilinear upsampling with lateral connections, the new high-resolution feature pyramid starts directly from 1 / 4 resolution, which is the highest resolution feature in the network.

[0092] The ability to build a new high-resolution feature pyramid directly from 1 / 4 resolution mainly relies on the design features of HigherHRNet as the CNN backbone network and the feature extraction and fusion strategy. Among them, HigherHRNet is a network structure designed specifically for high-resolution feature extraction. It can maintain high-resolution representation throughout the network and enhance feature representation through a multi-scale fusion mechanism. In HigherHRNet, branches of different resolutions are processed in parallel and information is exchanged through specific fusion modules.

[0093] Specifically, the input image is resized to a size suitable for network processing and used as the input of HigherHRNet. Since the goal is to build a feature pyramid starting from 1 / 4 resolution, no additional downsampling of the input image is required at this step. Then, initial feature extraction is performed, and the initial layers of HigherHRNet, such as convolutional layers and possible pooling layers, directly process the input image to generate preliminary feature maps. The resolution of these feature maps is usually similar to that of the input image, but may be slightly reduced due to reasons such as the step size of the convolution kernel. However, in the design of HigherHRNet, excessive downsampling in the initial stage is avoided as much as possible to retain more detailed information. In HigherHRNet, branches of different resolutions, such as 1 / 4, 1 / 8, 1 / 16, etc., process feature maps in parallel. However, unlike traditional feature pyramids, HigherHRNet builds feature pyramids directly from higher resolutions, such as 1 / 4. This is achieved by setting the output of the first feature extraction stage directly to a feature map with 1 / 4 resolution during network design. Next, other branches in the network, such as 1 / 8, 1 / 16, etc., generate feature maps with lower resolutions through gradual downsampling. However, these low-resolution feature maps interact and fuse with high-resolution feature maps to ensure that the feature representations throughout the network are multi-scale and high-resolution. HigherHRNet achieves information interaction and fusion between feature maps of different resolutions through mechanisms such as exchange units. This multi-scale fusion strategy not only enhances the richness of feature representation, but also enables the network to capture the characteristics of skin lesion areas at different scales.

[0094] The HigherHRNet backbone network is mainly responsible for extracting multi-scale high-resolution features of the image. The network maintains high-resolution feature maps from input to output, and realizes information interaction and fusion between feature maps of different resolutions through exchange modules. Then the branch network is connected. After the high-resolution feature pyramid of HigherHRNet is built, the lesion area regression network branch and the lesion area segmentation network branch will be connected to the corresponding feature map for processing. These branch networks will further extract and process features according to their respective task requirements, and output the final prediction results, that is, further extract the features of the lesion area and output the coordinate values ​​of the detection box.

[0095] As an example, suppose a photo of a foot lesion is taken, which contains some tiny lesion areas, which may have different shapes and sizes. In a traditional feature pyramid, the image is first downsampled to a lower resolution, then features are extracted through a series of convolutional layers, and the resolution is restored by upsampling when necessary. However, in this process, the tiny lesion area may lose some detail information due to downsampling. In HigherHRNet, the network builds a feature pyramid starting from the high resolution of the input image and maintains the high-resolution feature map throughout the network. This means that at each layer of the network, we can obtain a feature map containing detailed information of the lesion area. For example, in the first layer of feature maps, the edges and textures of the lesion area on the foot can be clearly seen; in subsequent feature maps, although the features become more abstract, the contours and location information of the lesion area are still well preserved. In HigherHRNet, feature maps of different scales are also fused through exchange units. This fusion occurs not only between adjacent scales, but also across multiple scales. Suppose there are two scales of feature maps, one is a feature map A with high resolution but less semantic information, and the other is a feature map B with low resolution but rich semantic information. Through the exchange module, the high-resolution details in the feature map foot A can be fused with the low-resolution semantic information in the feature map foot B. In this way, a new feature map can be obtained, which contains both the detailed texture information of the skin lesion area (from A) and the high-level semantic information such as the category and location of the skin lesion area (from B). This multi-scale fusion strategy enables the network to capture the characteristics of the skin lesion area at different scales, thereby improving the accuracy and robustness of detection.

[0096] In this embodiment, a new high-resolution feature pyramid is used. This design breaks the pattern of the traditional feature pyramid starting from a lower resolution (such as 1 / 32) and gradually upsampling, but directly starts from the feature map of 1 / 4 resolution. This means that at an earlier stage of the network, higher spatial resolution information is retained, providing rich details for subsequent high-precision skin lesion area detection and segmentation. As a multi-scale high-resolution network, HigherHRNet can more effectively process and fuse feature information of different scales through its unique network structure. Combined with the new high-resolution feature pyramid, it can extract richer feature representations while maintaining high resolution. Constructing the feature pyramid directly from 1 / 4 resolution allows the network to access high-resolution input data earlier, thereby retaining more detailed information. This is particularly important for the detection of small-scale skin lesion areas, because small-scale targets are prone to lose key information at lower resolutions. The design of the high-resolution feature pyramid enables the network to learn feature representations at different scales, improving the network's adaptability to skin lesion areas of different sizes. This multi-scale feature learning capability enhances the robustness of the network, enabling it to maintain stable performance in the face of complex and changing skin lesion conditions. Although starting directly from high resolution may increase the amount of computation in the initial stage, since this high-resolution information can be more effectively utilized in subsequent stages, unnecessary computational redundancy may be reduced overall. In addition, the relationship between computational effort and performance can be further balanced by optimizing the network structure and parameter configuration.

[0097] In this embodiment, by directly building a high-resolution feature pyramid starting from 1 / 4 resolution, HigherHRNet can better retain the detail information in the input image, which is particularly important for the detection and segmentation of small-scale targets such as foot lesions. At the same time, the multi-scale fusion strategy enhances the robustness and performance of the network, allowing the entire network to maintain a high detection and segmentation accuracy in complex and changeable skin lesion situations.

[0098] In some optional implementations, the high-resolution feature pyramid is input into the branch network to obtain the skin lesion features corresponding to the skin lesion area and the coordinate values ​​corresponding to the skin lesion area, including:

[0099] The optimal feature map is selected from the high-resolution feature pyramid as the feature map for detecting the skin lesion area, wherein the optimal feature map is a feature map that meets the preset resolution and contains the most detailed features.

[0100] A plurality of aiming point frames of different sizes are set on the skin lesion area detection feature map to cover possible skin lesion areas.

[0101] Regression optimization is performed on the possible skin lesion area to obtain the coordinate values ​​corresponding to the skin lesion area.

[0102] Based on the skin lesion area detection feature map and the coordinate values ​​corresponding to the skin lesion area, the skin lesion features corresponding to the skin lesion area are extracted.

[0103] Reference Figure 2 The figure shows a schematic diagram of skin lesion area positioning. Skin lesion segmentation usually involves separating the skin lesion area in the image from the background. The main function of the skin lesion area regression network branch is to regress the coordinate values ​​of the skin lesion area detection box, and finally realize the skin lesion area positioning.

[0104] Specifically, the feature map most suitable for skin lesion area detection is selected from the high-resolution feature pyramid of HigherHRNet. Usually, a feature map with high resolution and rich detail information is selected as input. Then, in the network branch, a series of convolutional layers are used to further extract the features of the skin lesion area. Each convolutional layer is followed by an activation function such as ReLU to increase the nonlinear ability of the network. Secondly, drawing on the idea of ​​anchor boxes in target detection, multiple anchor boxes of different sizes and proportions are preset at each position of the feature map to cover possible skin lesion areas. Then the feature map is mapped to the coordinate value of the detection box through a fully connected layer or a convolutional layer. The coordinate value usually includes four parameters: center point coordinates, width and height, or a more complex parameterized representation is used, such as a rotation box. Then the loss function is calculated and optimized. The loss function can use smooth loss (Smooth L1 Loss) or IoU loss (Intersection over Union Loss), etc., to regress and optimize the coordinate value of the detection box. These loss functions can better handle the error of the coordinate value and improve the positioning accuracy of the detection box. The network parameters are updated using optimization algorithms such as SGD (stochastic gradient descent) and Adam. During the learning process, the gradient of the loss function is passed to each layer of the network through the back-propagation algorithm to optimize parameters such as weights and biases.

[0105] The following example illustrates that a customized regression network branch can be designed for the task requirements of foot lesion area detection frame regression. This branch can receive high-resolution feature maps from HigherHRNet as input, and further extract features of the lesion area through a series of convolutional layers and activation functions. In the regression network branch, anchor boxes can be used to predict the location and size of the lesion area. Each anchor box is preset with a certain shape and size to cover the possible lesion area. The network then learns how to adjust these anchor boxes to better match the actual lesion area. Specifically, the regression network branch outputs a series of parameters, such as center point coordinates, width, height, etc., which are used to describe the location and shape of the detection box. During the learning process, a loss function (such as Smooth L1 Loss or IoU Loss) is used to compare the difference between the predicted detection box and the actual lesion area, and the network parameters are optimized through the back propagation algorithm. Finally, the coordinate values ​​corresponding to the initial lesion area and the lesion features are determined through the network branch.

[0106] In this embodiment, the branch network can better adapt to the characteristic distribution and shape changes of the skin lesion area, thereby improving the regression accuracy of the detection frame. At the same time, since it directly receives the high-resolution feature map from HigherHRNet as input, it can make full use of the detailed information in the image to improve the accuracy of detection.

[0107] In some optional embodiments, the skin lesion recognition model is established by the following steps:

[0108] Obtain a sample dataset of skin lesion disease images. You can collect a large number of annotated skin lesion image datasets.

[0109] The skin lesion position in each skin lesion disease image in the sample data set is marked. In this embodiment, the skin lesion position can be marked by polygonal marking. Then the image is preprocessed as necessary, such as scaling, normalization, and enhancement (rotation, flipping, etc.) to ensure that the image size is consistent and can adapt to the neural network input requirements.

[0110] The labeled sample data set is divided into training set, validation set and test set.

[0111] An initial multi-task convolutional neural network is established, wherein the initial multi-task convolutional neural network includes a backbone network, a lesion region regression branch network, and a lesion region segmentation branch network. The backbone network includes multiple convolutional layers, pooling layers, and activation function layers of different scales connected in sequence. The lesion region regression branch network and the lesion region segmentation branch network are respectively connected to the output end of the backbone network. The lesion region regression branch network is used to locate the lesion region, and the lesion region segmentation branch network is used to segment the lesion region. The purpose of the lesion region regression branch network is to detect the specific location of the lesion on the sole of the foot. If there are multiple lesions, different lesion regions can be circled with a detection frame to determine the location information of multiple lesions. The purpose of the lesion region segmentation branch network is to segment the lesion region from the image.

[0112] The labeled sample dataset is input into the initial multi-task convolutional neural network for training to obtain a skin lesion recognition model.

[0113] In this embodiment, the backbone network includes a plurality of convolutional layers, pooling layers and activation function layers of different scales connected in sequence. Through a series of convolutional layers, the network can extract meaningful features from the input image. And each downsampling operation will reduce the size of the feature map, which helps to reduce the amount of calculation and speed up the training. In this embodiment, it is directly adopted from 1 / 4 resolution. As the resolution decreases, the physical space represented by each pixel increases, so that more contextual information can be captured. Different levels of downsampling provide feature representations of different scales, which is particularly useful for processing targets of various sizes. And the shallower downsampling layer retains more details, while the deeper layer provides a more abstract representation. This multi-level representation helps to capture features at different levels. In this architecture, the downsampling part is usually connected to the upsampling part through a skip connection, so that fine details can be restored in the output and the detection accuracy can be improved.

[0114] Among them, the loss function of the skin lesion area regression branch network is:

[0115]

[0116] Among them, y t Indicates the real coordinate value, y p represents the predicted value of the regression branch network of the skin lesion area, N represents the total number of samples, and i represents the i-th sample.

[0117] The loss function of the skin lesion area segmentation branch network is:

[0118] CEL=-[ylogp+(1-y)log(1-p)]

[0119] Among them, y represents the actual skin lesion area, which takes the value of 1 or 0, and p represents the probability that the skin lesion area segmentation branch network predicts that it belongs to the skin lesion area.

[0120] Use the training set data to train the model, optimize the model performance by adjusting the hyperparameters, and use the validation set data to adjust the model's hyperparameters to prevent overfitting. Pay attention to the changes in indicators during the training process, such as accuracy, recall, F1 score, etc. Then use an independent test set to evaluate the generalization ability of the model, and calculate the model's Dice coefficient, Jaccard index and other indicators to evaluate the segmentation effect. After training and testing the model, the model can be used to locate the lesion area on the foot, thereby obtaining the pressure site of the relevant airbag.

[0121] Both the skin lesion region regression branch network and the skin lesion region segmentation branch network can benefit from the high-resolution feature pyramid. The regression branch can more accurately predict the coordinate values ​​of the skin lesion region, while the segmentation branch can more finely divide the boundaries of the skin lesion region. The collaborative work of these two branches enables the entire network to show higher performance in skin lesion region detection and segmentation tasks.

[0122] In some optional embodiments, based on the skin lesion type, skin lesion monitoring data, and current parameters, the treatment data is determined, including:

[0123] Based on the type of skin lesions, the target pressure, target temperature and target humidity corresponding to the skin lesion area are determined.

[0124] Based on the skin lesion monitoring data, the current foot pressure, foot temperature and foot humidity are determined.

[0125] Based on the current parameters, a current airbag pressure, a current airbag temperature, and a current airbag humidity of each airbag of the airbag array in the airbag control system are determined.

[0126] The pressure to be adjusted for each airbag is determined based on the target pressure, the foot pressure, and the current airbag pressure.

[0127] The temperature to be adjusted for each airbag is determined based on the target temperature, the foot temperature, and the current airbag temperature.

[0128] Based on the target humidity, the foot humidity, and the current airbag humidity, the humidity to be adjusted for each airbag is determined.

[0129] Specifically, the pressure distribution data of the foot during walking or standing can be collected in real time by means of a pressure sensor array embedded in the insole or sole. The pressure sensor array, such as a pressure pad, collects pressure distribution images of the foot, which can be represented in grayscale or color form, where different colors or grayscale values ​​represent different pressure levels. The collected images are then preprocessed, including steps such as denoising, filtering, and contrast enhancement to improve image quality and reduce the difficulty of subsequent processing. Useful feature information is extracted from the preprocessed images. These features may include the shape, size, position, intensity, etc. of the pressure distribution. The specific features to be selected depend on the application scenario and recognition requirements. The extracted features are classified or recognized using machine learning or deep learning algorithms, which may include support vector machines, neural networks, decision trees, etc. The classification or recognition results are post-processed to improve accuracy and reliability. Post-processing includes steps such as removing noise points, smoothing, and filling missing data. Then, combined with the data from the pressure sensor array, the image information is fused with the pressure distribution data to accurately monitor the pressure distribution of various areas of the foot. Features such as pressure peaks and pressure distribution shapes are extracted from pressure sensor data, and classification or regression analysis is performed using machine learning or deep learning algorithms to assess foot pressure conditions. Combined with the data from the pressure sensor array, the image information is fused with the pressure distribution data to accurately monitor the pressure distribution in various areas of the foot. The system can also provide feedback or suggestions to users based on the analysis results, such as adjusting walking posture and choosing appropriate shoes, by judging the force applied to the foot and whether there are abnormal pressure points.

[0130] In addition, infrared thermal imaging technology can be used to obtain foot temperature distribution images. Abnormal temperature areas can be extracted through image processing algorithms such as threshold segmentation and edge detection. The temperature data can also be further analyzed in combination with the foot thermal model to evaluate foot blood circulation, inflammation, etc.

[0131] Regarding foot humidity, the humidity sensor data can be directly read and simple threshold judgment or trend analysis can be performed to monitor the humidity changes in the foot environment. Furthermore, the humidity data can be integrated with the pressure and temperature data for a more comprehensive assessment of foot health.

[0132] Then, according to the current airbag parameters of the airbag control system and the target parameters determined according to the diagnosis results, namely the target pressure, target temperature and target humidity, the parameters that need to be adjusted for the airbag, namely the current airbag pressure, current airbag temperature and current airbag humidity, can be determined. In this way, the parts of the foot that need to be pressurized can be accurately determined, and precise treatment can be given to the parts that need to be treated.

[0133] In some optional embodiments, after determining the skin lesion characteristics and skin lesion types, the method further includes:

[0134] Upload skin lesion characteristics and types to the cloud-based medical knowledge base.

[0135] Based on the skin lesion characteristics, skin lesion types and cloud-based medical knowledge base, the severity of the skin lesions is assessed and an assessment result is generated.

[0136] The evaluation results are sent to the user terminal for visual display.

[0137] After using deep learning algorithms to extract features and classify skin lesion images and determine the type of skin lesions, such as foot surgery, trauma, diabetic foot and other foot diseases, the cloud-based medical knowledge base and expert experience can be combined to evaluate the severity of the skin lesions, analyze the changing trend of the skin lesions, and predict their possible development paths. The evaluation results are then presented to patients or doctors in an intuitive way through mobile APPs, web pages and other channels. Detailed explanations and suggestions are provided to help patients or doctors understand the progress of treatment or care and the next plan. Furthermore, targeted treatment or care plans can be formulated based on the patient's skin lesion condition, physical condition, living habits and other factors.

[0138] The identification results, type, severity and other information of the skin lesions can be displayed on the screen. Auxiliary information such as magnified pictures and comparison pictures of the skin lesions can be provided to help patients or doctors understand the skin lesions more clearly. Prompts such as sound or light signals can be issued to ensure that patients or doctors can pay attention to the analysis results in time.

[0139] Furthermore, skin lesion images and sensor data are collected regularly, compared with the data collected last time, and the changes in skin lesions (such as shrinkage in area, lightening in color, etc.) are evaluated to determine the effect of treatment or care. The treatment plan or care plan is then adjusted based on the evaluation results to provide a basis for subsequent plan adjustments.

[0140] In this embodiment, a foot disease treatment device based on image recognition is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0141] This embodiment provides a foot disease treatment device based on image recognition, such as Figure 3 As shown, the device comprises:

[0142] An image acquisition module 201 is used to acquire a foot image to be analyzed;

[0143] An identification module 202 is used to identify the skin lesion area in the foot image to be analyzed and determine the skin lesion characteristics of the skin lesion area;

[0144] A determination module 203, for determining the type of skin lesions based on the characteristics of the skin lesions;

[0145] The data acquisition model 204 is used to acquire skin lesion monitoring data and current parameters of the airbag control system. The skin lesion monitoring data is detected and acquired by a sensor arranged near the skin lesion area. The airbag control system includes an airbag array, which is used to treat the foot. The parameters include the pressure, temperature and humidity of each airbag.

[0146] A conversion module 205, for determining treatment data based on skin lesion type, skin lesion monitoring data, and current parameters;

[0147] The communication module 206 is used to convert the treatment data into a control signal and transmit it to the airbag control system. The airbag control system is used to adjust the parameters of each airbag according to the received control signal.

[0148] In some optional implementations, the identification module 202 includes:

[0149] An input unit inputs the foot image to be analyzed into a skin lesion recognition model, which includes a backbone network and a branch network, wherein the backbone network constructs a high-resolution feature pyramid starting from a 1 / 4 resolution;

[0150] The backbone network learning unit is used to pass the foot image to be analyzed through the backbone network to obtain a high-resolution feature pyramid composed of feature maps of different resolutions;

[0151] The branch network learning unit is used to input the high-resolution feature pyramid into the branch network to obtain the skin lesion features corresponding to the skin lesion area and the coordinate values ​​corresponding to the skin lesion area.

[0152] In some optional implementations, the branch network learning unit includes:

[0153] The regression subunit is used to select the optimal feature map from the high-resolution feature pyramid as the skin lesion area detection feature map, wherein the optimal feature map is the feature map that meets the preset resolution and contains the most detailed features; multiple aiming point frames of different sizes are set on the skin lesion area detection feature map to cover the possible skin lesion area; regression optimization is performed on the possible skin lesion area to obtain the coordinate value corresponding to the skin lesion area; based on the skin lesion area detection feature map and the coordinate value corresponding to the skin lesion area, the skin lesion features corresponding to the skin lesion area are extracted.

[0154] In some optional embodiments, the device further comprises:

[0155] A model building module is used to obtain a sample data set of skin lesion disease images; annotate the lesion position in each skin lesion disease image in the sample data set; divide the annotated sample data set into a training set, a validation set and a test set; establish an initial multi-task convolutional neural network, wherein the initial multi-task convolutional neural network includes a backbone network, a skin lesion area regression branch network, and a skin lesion area segmentation branch network, the backbone network includes a plurality of convolutional layers, pooling layers and activation function layers of different scales connected in sequence, the skin lesion area regression branch network and the skin lesion area segmentation branch network are respectively connected to the output end of the backbone network, the skin lesion area regression branch network is used for lesion area positioning, and the skin lesion area segmentation branch network is used for segmenting the skin lesion area; the annotated sample data set is input into the initial multi-task convolutional neural network for training to obtain a skin lesion recognition model.

[0156] The image recognition-based foot disease treatment device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0157] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0158] The embodiment of the present invention also provides a computer device having the above Figure 3 The image recognition-based foot disease treatment device shown.

[0159] See also Figure 4 , Figure 4 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 4 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0160] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0161] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0162] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0163] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0164] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0165] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0166] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0167] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An airbag control method based on image recognition, characterized in that: The method comprises: Acquire a foot image to be analyzed; Identify the skin lesion area in the foot image to be analyzed, and determine the skin lesion features of the skin lesion area; specifically include: input the foot image to be analyzed into a skin lesion recognition model, the skin lesion recognition model includes a trunk network and a branch network, wherein the trunk network constructs a high-resolution feature pyramid starting from a 1 / 4 resolution; the foot image to be analyzed passes through the trunk network to obtain the high-resolution feature pyramid composed of feature maps of different resolutions; select the optimal feature map from the high-resolution feature pyramid as the skin lesion area detection feature map, wherein the optimal feature map is a feature map that meets the preset resolution and contains the most detailed features; set multiple aiming point frames of different sizes on the skin lesion area detection feature map to cover possible skin lesion areas; perform regression optimization on the possible skin lesion area to obtain the coordinate value corresponding to the skin lesion area; extract the skin lesion features corresponding to the skin lesion area based on the skin lesion area detection feature map and the coordinate value corresponding to the skin lesion area; Determining the type of skin lesions based on the characteristics of the skin lesions; Acquiring skin lesion monitoring data and current parameters of an airbag control system, wherein the skin lesion monitoring data is obtained by detection by a sensor disposed near the skin lesion area, the airbag control system includes an airbag array, and the parameters include pressure, temperature, and humidity of each airbag; Determining treatment data based on the skin lesion type, the skin lesion monitoring data, and the current parameters; The treatment data is converted into a control signal and transmitted to the airbag control system, and the airbag control system is used to adjust the parameters of each airbag according to the received control signal.

2. The method according to claim 1, characterized in that The skin lesion recognition model is established by the following steps: Get a sample dataset of skin lesion disease images; Annotating the skin lesion position in each skin lesion disease image in the sample data set; Dividing the labeled sample data set into a training set, a validation set, and a test set; Establishing an initial multi-task convolutional neural network, wherein the initial multi-task convolutional neural network includes a backbone network, a skin lesion region regression branch network, and a skin lesion region segmentation branch network, the backbone network includes a plurality of convolutional layers, pooling layers, and activation function layers of different scales connected in sequence, the skin lesion region regression branch network and the skin lesion region segmentation branch network are respectively connected to the output end of the backbone network, the skin lesion region regression branch network is used for skin lesion region positioning, and the skin lesion region segmentation branch network is used for segmenting the skin lesion region; The labeled sample data set is input into the initial multi-task convolutional neural network for training to obtain the skin lesion recognition model.

3. The method according to claim 1, characterized in that The determining of treatment data based on the skin lesion type, the skin lesion monitoring data, and the current parameters includes: Based on the skin lesion type, determining a target pressure, a target temperature, and a target humidity corresponding to the skin lesion area; Based on the skin lesion monitoring data, determining current foot pressure, foot temperature and foot humidity; Based on the current parameters, determine the current airbag pressure, current airbag temperature and current airbag humidity of each airbag in the airbag array of the airbag control system; Determining the pressure to be adjusted for each airbag based on the target pressure, the foot pressure, and the current airbag pressure; Determining a temperature to be adjusted for each airbag based on the target temperature, the foot temperature, and the current airbag temperature; The humidity to be adjusted for each airbag is determined based on the target humidity, the foot humidity, and the current airbag humidity.

4. The method according to claim 1, characterized in that: After determining the skin lesion characteristics and the skin lesion type, the method further includes: Uploading the skin lesion characteristics and the skin lesion type to a medical knowledge base; Based on the skin lesion characteristics, the skin lesion type and the medical knowledge base, assess the severity of the skin lesion and generate an assessment result; The evaluation result is sent to a user terminal for visual display.

5. An airbag control device based on image recognition, characterized in that: The device comprises: An image acquisition module, used for acquiring a foot image to be analyzed; an identification module, used for identifying a skin lesion area in the foot image to be analyzed, and determining skin lesion features of the skin lesion area; A determination module, used for determining the type of skin lesions based on the skin lesion characteristics; A data acquisition model, used to acquire skin lesion monitoring data and current parameters of an airbag control system, wherein the skin lesion monitoring data is detected and acquired by a sensor disposed near the skin lesion area, the airbag control system includes an airbag array, and the parameters include pressure, temperature and humidity of each airbag; a conversion module, configured to determine treatment data based on the skin lesion type, the skin lesion monitoring data, and the current parameters; A communication module, used for converting treatment data into control signals and transmitting them to the airbag control system, wherein the airbag control system is used for adjusting the parameters of each airbag according to the received control signals; The identification module comprises: An input unit, used for inputting the foot image to be analyzed into a skin lesion recognition model, wherein the skin lesion recognition model includes a trunk network and a branch network, wherein the trunk network constructs a high-resolution feature pyramid starting from a 1 / 4 resolution; a trunk network learning unit, used for passing the foot image to be analyzed through the trunk network to obtain the high-resolution feature pyramid composed of feature maps of different resolutions; and a branch network learning unit, used for inputting the high-resolution feature pyramid into the branch network to obtain skin lesion features corresponding to the skin lesion area and coordinate values ​​corresponding to the skin lesion area; The branch network learning unit includes: a regression subunit, which is used to select an optimal feature map from the high-resolution feature pyramid as a skin lesion area detection feature map, wherein the optimal feature map is a feature map that meets a preset resolution and contains the most detail features; multiple aiming point frames of different sizes are set on the skin lesion area detection feature map to cover possible skin lesion areas; regression optimization is performed on the possible skin lesion area to obtain coordinate values ​​corresponding to the skin lesion area; and skin lesion features corresponding to the skin lesion area are extracted based on the skin lesion area detection feature map and the coordinate values ​​corresponding to the skin lesion area.

6. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the image recognition-based airbag control method according to any one of claims 1 to 4 by executing the computer instructions.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the airbag control method based on image recognition as described in any one of claims 1-4.

8. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to enable a computer to execute the airbag control method based on image recognition according to any one of claims 1 to 4.

Images