Skin detection method for an object, storage medium and processor

Abstract

The application discloses a skin detection method of an object, a storage medium and a processor. The method comprises the following steps: acquiring a skin image covering an outer surface of an object to be detected; identifying the skin image to determine a skin lesion area of the object to be detected, wherein the skin lesion area is an image area with a skin lesion feature in the skin image; determining a skin lesion attribute of the object to be detected based on the skin lesion feature of the skin lesion area and an object type of the object to be detected, wherein the skin lesion attribute is used for describing a skin lesion generated by the object to be detected; and matching, based on the skin lesion attribute, lesion data recorded in a lesion database to determine a pathological result of the object to be detected. The image detection method provided by the application solves the technical problem of low detection precision of the skin image of the object.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer, in particular to a skin detection method of an object, a storage medium and a processor. BACKGROUND

[0002] At present, for artificial intelligence assisted skin disease diagnosis, it is usually based on deep learning to realize the classification prediction of skin diseases, so as to determine the disease diagnosis result, but this method is too rough and cannot accurately judge the disease, thereby there is a technical problem of low detection precision of the skin image of the object.

[0003] At present, there is no effective solution to the above problems. SUMMARY

[0004] The embodiments of the present application provide a skin detection method of an object, a storage medium and a processor, which at least solve the technical problem of low detection precision of the skin image of the object.

[0005] According to an aspect of the embodiments of the present application, a skin detection method of an object is provided, comprising: acquiring a skin image covering the outer surface of a to-be-detected object; identifying the skin image to determine a lesion area of the to-be-detected object, wherein the lesion area is an image area in the skin image that has a lesion feature; determining a lesion attribute of the to-be-detected object based on the lesion feature of the lesion area and the object type of the to-be-detected object, wherein the lesion attribute is used to describe the lesion generated by the to-be-detected object; and matching the lesion attribute with lesion data recorded in a lesion database to determine a pathological result of the to-be-detected object.

[0006] According to another aspect of the embodiments of the present application, a skin detection method of an object is also provided, comprising: in response to an image input instruction acting on an operation interface, displaying a skin image covering the outer surface of a to-be-detected object from a medical diagnosis platform on the operation interface; and in response to a detection operation instruction acting on the operation interface, displaying a pathological result of the to-be-detected object on the operation interface, wherein the pathological result is obtained by matching the lesion attribute of the to-be-detected object with lesion data recorded in a lesion database, the lesion attribute is determined based on the lesion feature of the lesion area and the object type of the to-be-detected object, and the lesion area is obtained by identifying the skin image.

[0007] According to another aspect of the embodiments of the present application, a skin detection method for an object is also provided, including: displaying a skin image overlaid on an outer surface of an object to be detected on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device; identifying the skin image to determine a skin lesion area of the object to be detected, wherein the skin lesion area is an image area in the skin image sensed by the VR device or the AR device, in which a skin lesion feature exists; determining a skin lesion attribute of the object to be detected based on the skin lesion feature of the skin lesion area and an object type of the object to be detected, wherein the skin lesion attribute is used to describe a skin lesion generated by the object to be detected; matching the skin lesion attribute with lesion data recorded in a lesion database to determine a pathological result of the object to be detected; and driving the VR device or the AR device to display the skin lesion attribute and the pathological result.

[0008] According to another aspect of the embodiments of the present application, a skin detection method for an object is also provided, including: displaying a skin image overlaid on an outer surface of an object to be detected on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device; identifying the skin image to determine a skin lesion area of the object to be detected, wherein the skin lesion area is an image area in the skin image sensed by the VR device or the AR device, in which a skin lesion feature exists; determining a skin lesion attribute of the object to be detected based on the skin lesion feature of the skin lesion area and an object type of the object to be detected, wherein the skin lesion attribute is used to describe a skin lesion generated by the object to be detected; matching the skin lesion attribute with lesion data recorded in a lesion database to determine a pathological result of the object to be detected; and driving the VR device or the AR device to display the skin lesion attribute and the pathological result.

[0009] According to another aspect of the embodiments of the present application, a skin detection method for an object is also provided, including: displaying a skin image overlaid on an outer surface of an object to be detected on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device; identifying the skin image to determine a skin lesion area of the object to be detected, wherein the skin lesion area is an image area in the skin image sensed by the VR device or the AR device, in which a skin lesion feature exists; determining a skin lesion attribute of the object to be detected based on the skin lesion feature of the skin lesion area and an object type of the object to be detected, wherein the skin lesion attribute is used to describe a skin lesion generated by the object to be detected; matching the skin lesion attribute with lesion data recorded in a lesion database to determine a pathological result of the object to be detected; and driving the VR device or the AR device to display the skin lesion attribute and the pathological result.

[0010] The embodiment of the present application further provides a skin detection device for an object, comprising: a first display unit configured to display a skin image overlaid on an outer surface of a to-be-detected object from a medical diagnosis platform on an operation interface in response to an image input instruction acting on the operation interface; and a second display unit configured to display a pathological result of the to-be-detected object on the operation interface in response to a detection operation instruction acting on the operation interface, wherein the pathological result is obtained by matching lesion data recorded in a lesion database based on a skin lesion attribute of the to-be-detected object, and the skin lesion attribute is determined based on a skin lesion feature of a skin lesion area and an object type of the to-be-detected object, and the skin lesion area is obtained by recognizing the skin image.

[0011] The embodiment of the present application further provides a skin detection device for an object, comprising: a display unit configured to display a skin image overlaid on an outer surface of a to-be-detected object on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device; a third determination unit configured to recognize the skin image and determine a skin lesion area of the to-be-detected object, wherein the skin lesion area is an image area in which a skin lesion feature exists in the skin image sensed by the VR device or the AR device; a fourth determination unit configured to determine a skin lesion attribute of the to-be-detected object based on the skin lesion feature of the skin lesion area and an object type of the to-be-detected object, wherein the skin lesion attribute is used to describe a skin lesion generated by the to-be-detected object; a second matching unit configured to match lesion data recorded in a lesion database based on the skin lesion attribute and determine a pathological result of the to-be-detected object; and a driving unit configured to drive the VR device or the AR device to display the skin lesion attribute and the pathological result.

[0012] The embodiment of the present application further provides a skin detection device for an object, comprising: a first calling unit configured to obtain a skin image overlaid on an outer surface of a to-be-detected object by calling a first interface, wherein the first interface comprises a first parameter, and a parameter value of the first parameter is the skin image; a fifth determination unit configured to recognize the skin image and determine a skin lesion area of the to-be-detected object, wherein the skin lesion area is an image area in which a skin lesion feature exists in the skin image; a sixth determination unit configured to determine a skin lesion attribute of the to-be-detected object based on the skin lesion feature of the skin lesion area and an object type of the to-be-detected object, wherein the skin lesion attribute is used to describe a skin lesion generated by the to-be-detected object; a third matching unit configured to match lesion data recorded in a lesion database based on the skin lesion attribute and determine a pathological result of the to-be-detected object; and a second calling unit configured to output the skin lesion attribute and the pathological result by calling a second interface, wherein the second interface comprises a second parameter, and a value of the second parameter is the skin lesion attribute and the pathological result.

[0013] The embodiment of the present application further provides a computer readable storage medium. The computer readable storage medium comprises a stored program, wherein the program controls a device where the computer readable storage medium is located to execute the skin detection method for an object of the embodiment of the present application when the program is run by a processor.

[0014] The embodiment of the present application further provides a processor. The processor is used for running a program, wherein the program is used for executing the skin detection method of the object according to the embodiment of the present application.

[0015] In the embodiment of the present application, a skin image covering an outer surface of the object to be detected is acquired; the skin image is recognized to determine a lesion area of the object to be detected, wherein the lesion area is an image area in which a lesion feature exists in the skin image; based on the lesion feature of the lesion area and the object type of the object to be detected, a lesion attribute of the object to be detected is determined, wherein the lesion attribute is used for describing a lesion generated by the object to be detected; based on the lesion attribute, lesion data recorded in a lesion database is matched to determine a pathological result of the object to be detected. That is, the embodiment of the present application extracts the lesion area by processing the acquired skin image, matches the lesion attribute of the determined lesion area with the lesion data recorded in the lesion database, determines the attribute feature of the disease, and achieves the purpose of accurately determining the pathological result, thereby realizing the technical effect of improving the detection precision of the skin image of the object, and further the image detection method provided by the embodiment of the present application solves the technical problem of low detection precision of the skin image of the object. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are included to provide a further understanding of the present application and constitute a part of this application, illustrate certain illustrative embodiments of the present application and together with the description serve to explain the present application. In the drawings:

[0017] Figure 1 FIG. 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing the skin detection method of the object according to the embodiment of the present application;

[0018] Figure 2 FIG. 2 is a hardware structure block diagram of a virtual reality device for implementing the skin detection method of the object according to the embodiment of the present application;

[0019] Figure 3 FIG. 3 is a flowchart of the skin detection method of the object according to the embodiment of the present application;

[0020] Figure 4 FIG. 4 is a flowchart of another skin detection method of the object according to the embodiment of the present application;

[0021] Figure 5 FIG. 5 is a flowchart of another skin detection method of the object according to the embodiment of the present application;

[0022] Figure 6 FIG. 6 is a flowchart of another skin detection method of the object according to the embodiment of the present application;

[0023] Figure 7 is a flow chart of a clinical dermatoscope diagnosis method based on a transformer model according to an embodiment of the present application;

[0024] Figure 8 is a schematic diagram of a general dermatosis automatic diagnosis system according to an embodiment of the present application;

[0025] Figure 9 is a schematic diagram of a skin detection device of an object according to an embodiment of the present application;

[0026] Figure 10 is a schematic diagram of another skin detection device of an object according to an embodiment of the present application;

[0027] Figure 11 is a schematic diagram of another skin detection device of an object according to an embodiment of the present application;

[0028] Figure 12 is a schematic diagram of another skin detection device of an object according to an embodiment of the present application;

[0029] Figure 13 is a structural block diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0031] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0032] First, some of the nouns or terms appearing in the description of the embodiments of the present application are explained as follows:

[0033] DermatFormer, which can be used to perform a skin detection method of an object provided by the embodiments of the present application;

[0034] Token, a region label of an image after processing;

[0035] LTS (Lesion Token Selection), which can be used to automatically extract a label sequence of a lesion region to remove redundant features / guide an encoder of a visual transformer to select localized labels related to a lesion at different levels;

[0036] ViT (Vision Transformer), which can be used to divide an input image into multiple local image blocks, linearly embed each local image block, and send the obtained vector sequence to a standard encoder for learning;

[0037] CAD (Computer Assisted Diagnosis), which can be used to assist doctors in efficient diagnosis and accurate interpretation of the disease;

[0038] MSA (Multihead Self Attention), which can be used to generate an image feature sequence based on a label sequence;

[0039] CFM (Contextual Fusion Module), which can be used to fuse global context information;

[0040] LFM (Local Fusion Module), which can be used to fuse fine-grained local features;

[0041] BPD (Bilateral Prediction Distillation), which can be used to correct misclassified samples by using a pre-computed co-occurrence matrix of disease types and lesion attributes.

[0042] Embodiment 1

[0043] According to an embodiment of the present application, a method for detecting skin of an object is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] Figure 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for detecting skin of an object according to an embodiment of the present application. As shown in Figure 1 , the computer terminal 10 (or mobile device 10) can include one or more processors (processors can include but not limited to processing devices such as microprocessor MCU or programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports in the BUS bus), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 can include more or less components than those shown in Figure 1 , or have a different configuration from that shown in Figure 1 .

[0045] It should be noted that the one or more processors and / or other skin detection circuits of the object described above can be referred to herein as "skin detection circuits of the object". The skin detection circuits of the object can be embodied in whole or in part as software, hardware, firmware or any other combination. In addition, the skin detection circuits of the object can be a single independent processing module, or all or part of any one of the other elements combined into the computer terminal 10 (or mobile device). As referred to in the embodiments of the present application, the skin detection circuits of the object as a processor control (for example, the selection of the variable resistance terminal path connected with the interface).

[0046] The memory 104 can be used to store software programs of application software and modules, such as program instructions / data storage means corresponding to the skin detection method of the object in the embodiments of the present application, and the processor executes various functional applications and the skin detection method of the object by running the software programs and modules stored in the memory 104. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor, which can be connected to the computer terminal 10 through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0047] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network can include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (NIC) which can be connected to other network devices through a base station so as to be able to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module used to communicate with the Internet in a wireless manner.

[0048] The display can be, for example, a touch screen type liquid crystal display (LCD) which can enable a user to interact with the user interface of the computer terminal 10 (or mobile device).

[0049] Figure 2 is a hardware structure block diagram of a virtual reality device for implementing the skin detection method of the object according to the embodiments of the present application. As shown in Figure 2 The virtual reality device 204 is connected to the terminal 206, and the terminal 206 is connected to the server 202 through a network. The above-mentioned virtual reality device 204 is not limited to a virtual reality helmet, a virtual reality glasses, a virtual reality all-in-one machine, etc., the above-mentioned terminal 206 is not limited to a PC, a mobile phone, a tablet computer, etc., and the server 202 can be a server corresponding to a media file operator. The above-mentioned network includes but is not limited to a wide area network, a metropolitan area network, or a local area network.

[0050] Optionally, the virtual reality device 204 of the embodiment comprises a memory, a processor and a transmission device. The memory is configured to store an application program, which can be used to perform the following steps: obtaining a skin image overlaid on the outer surface of the object to be detected; identifying the skin image to determine a lesion area of the object to be detected, wherein the lesion area is an image area in which a lesion feature exists in the skin image; determining a lesion attribute of the object to be detected based on the lesion feature of the lesion area and the object type of the object to be detected, wherein the lesion attribute is used to describe the lesion generated by the object to be detected; and matching the lesion attribute with lesion data recorded in a lesion database to determine a pathological result of the object to be detected, thereby solving the technical problem of low detection accuracy of the skin image of the object and achieving the technical effect of improving the detection accuracy of the skin image of the object.

[0051] The terminal of the embodiment can be used to perform the following steps: displaying a skin image overlaid on the outer surface of an object to be detected on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device; identifying the skin image to determine a lesion area of the object to be detected, wherein the lesion area is an image area in which a lesion feature exists in the skin image sensed by the virtual reality device 204; determining a lesion attribute of the object to be detected based on the lesion feature of the lesion area and the object type of the object to be detected, wherein the lesion attribute is used to describe the lesion generated by the object to be detected; matching the lesion attribute with lesion data recorded in a lesion database to determine a pathological result of the object to be detected; and driving the VR device or the AR device / virtual reality device 204 to display the lesion attribute and the pathological result.

[0052] Optionally, the virtual reality device 204 of the embodiment is provided with an eye tracking head-mounted display (HMD) and an eye tracking module, which have the same functions as those in the above-mentioned embodiments, i.e., the screen in the HMD head set is used to display real-time images, and the eye tracking module in the HMD is used to obtain the real-time motion path of the user's eyes. The terminal of the embodiment obtains the position information and motion information of the user in the real three-dimensional space through the tracking system, and calculates the three-dimensional coordinates of the user's head in the virtual three-dimensional space and the direction of the user's field of view in the virtual three-dimensional space.

[0053] Figure 2 The hardware structure diagram shown can not only be used as an exemplary block diagram of the above-mentioned AR / VR device (or mobile device), but also as an exemplary block diagram of the above-mentioned server.

[0054] In the above-mentioned operating environment, the present application provides a method for detecting a skin image of an object to be detected, comprising the following steps: Figure 3The skin detection method of the object shown. It should be noted that the skin detection method of the object of this embodiment can be implemented by Figure 1 The mobile terminal of the embodiment shown is executed.

[0055] Figure 3 is a flowchart of a skin detection method of an object according to an embodiment of the present application. As shown in Figure 3 The method can include the following steps:

[0056] Step S302, acquiring a skin image covering the outer surface of the object to be detected.

[0057] In the technical solution provided by the above step S302 of the present application, the skin image covering the outer surface of the object to be detected is acquired, wherein the object to be detected can be patients of different age groups, such as infants or the elderly, which are only used as examples and are not limited.

[0058] For example, the skin image covering the outer surface of the object to be detected can be a skin image uploaded by the patient on the client, or a skin image obtained by the dermatologist in clinical practice.

[0059] Step S304, identifying the skin image to determine the skin lesion area of the object to be detected, wherein the skin lesion area is an image area in the skin image that has a skin lesion feature.

[0060] In the technical solution provided by the above step S304 of the present application, the acquired skin image is identified to determine the skin lesion area of the object to be detected, wherein the skin lesion area can be an image area in the skin image that has a skin lesion feature, such as a skin lesion area, a skin lesion area, a lesion area, a skin lesion target area, a lesion area, etc., which are only used as examples and are not limited; the skin lesion feature can be the image feature after removing the redundancy in the skin image.

[0061] Step S306, determining the skin lesion attribute of the object to be detected based on the skin lesion feature of the skin lesion area and the object type of the object to be detected, wherein the skin lesion attribute is used to describe the skin lesion generated by the object to be detected.

[0062] In the technical solution provided in the step S306 of the present application, according to the obtained skin lesion area of the to-be-detected object, the skin lesion feature of the skin lesion area is obtained, and the skin lesion attribute of the skin lesion area is determined in combination with the type of the to-be-detected object, wherein the skin lesion attribute can be the attribute feature of the pathological lesion, and can serve as the evidence of the explainability of the disease result of the to-be-detected object, for example, the skin lesion attribute can include redness, dandruff, and pigmentation, etc., which are only used for illustration and are not limited in particular; the object type can be the skin of different body parts, for example, the skin of different body parts can be the skin of legs and the skin of face, which are only used for illustration and are not limited in particular.

[0063] In step S308, the skin lesion attribute of the to-be-detected object is matched with the pathological data recorded in the pathological database to determine the pathological result of the to-be-detected object.

[0064] In the technical solution provided in the step S308 of the present application, the determined skin lesion attribute of the to-be-detected object is matched with the obtained pathological data in the pathological database to determine the pathological result of the to-be-detected object, wherein the pathological result of the to-be-detected object can be the result of the disease diagnosis of the skin lesion area of the to-be-detected object, for example, the pathological result can be the pathological type and the dermatosis category, etc., which are only used for illustration and are not limited in particular; the pathological database can be a statistical database for storing the pathological type and the skin lesion attribute set, for example, it can be the pathological data set library after the prior knowledge of the dermatologist is calibrated, which is only used for illustration and is not limited in particular.

[0065] Through the steps S302 to S308 of the present application, the skin image covering the outer surface of the to-be-detected object is obtained; the skin image is recognized to determine the skin lesion area of the to-be-detected object, wherein the skin lesion area is the image area with the skin lesion feature in the skin image; based on the skin lesion feature of the skin lesion area and the object type of the to-be-detected object, the skin lesion attribute of the to-be-detected object is determined, wherein the skin lesion attribute is used to describe the skin lesion generated by the to-be-detected object; based on the skin lesion attribute, the pathological data recorded in the pathological database is matched to determine the pathological result of the to-be-detected object. That is, the present embodiment extracts the skin lesion area by processing the obtained skin image, matches the skin lesion attribute of the determined skin lesion area with the pathological data recorded in the pathological database to determine the attribute feature of the disease, so as to accurately determine the pathological result, thereby realizing the technical effect of improving the detection precision of the skin image of the object, and further, the image detection method provided by the present embodiment solves the technical problem of low detection precision of the skin image of the object.

[0066] The above method of the embodiment will be further introduced below.

[0067] As an optional implementation, in step S304, the skin image is recognized to determine the skin lesion area of the to-be-detected object, including: recognizing the skin image to obtain a region identification sequence, where each region identification in the region identification sequence represents a skin area of the to-be-detected object; and extracting a sub-region identification sequence from the region identification sequence, where each region identification in the sub-region identification sequence represents a sub-skin lesion area in the skin lesion area.

[0068] In this embodiment, the obtained skin image is recognized to obtain a region identification sequence of the skin image, a sub-region identification sequence is extracted from the obtained region identification sequence, and a sub-skin lesion area is determined based on each region identification in the extracted sub-region identification sequence, so as to determine the skin lesion area of the to-be-detected object. The region identification sequence can be a local marker sequence (which can be referred to as a region marker sequence) obtained based on the obtained skin image, each region identification in the region identification sequence can represent a skin area of the to-be-detected object, for example, the region identification in the region identification sequence can be a local marker obtained based on the obtained skin image. The skin area can be represented by a position vector and a position code of the region identification. The sub-region identification sequence can be a part of the region identification sequence, for example, the sub-region identification sequence can be a marker result obtained by removing redundant features from the marker sequence of the skin lesion area. Each region identification in the sub-region identification sequence represents a sub-skin lesion area in the skin lesion area, for example, the region identification in the sub-region identification sequence can be a local marker related to the lesion.

[0069] As an optional implementation, the sub-region identification sequence is extracted from the region identification sequence, including: converting the region identification sequence into an image feature sequence of the skin image, where each image feature in the image feature sequence represents a skin area corresponding to the region identification in the region identification sequence; determining a sub-image feature sequence corresponding to the skin lesion area from the image feature sequence, where the skin lesion area includes a skin area corresponding to the image feature in the sub-image feature sequence; and determining the sub-region identification sequence corresponding to the sub-image feature sequence.

[0070] In this embodiment, based on the obtained region identification sequence of the skin image, a corresponding image feature sequence of the skin image is generated, a sub-image feature sequence corresponding to the skin lesion area is determined from the generated image feature sequence, and a sub-region identification sequence corresponding to the sub-image feature sequence is determined, where each image feature in the image feature sequence can represent a skin area corresponding to the region identification in the region identification sequence, and the skin lesion area can include a skin area corresponding to the image feature in the sub-image feature sequence.

[0071] Optionally, the region identification sequence of the skin image can be generated by the multi-head self-attention module from the image feature sequence of the corresponding skin image, and the sub-image feature sequence corresponding to the lesion region can be extracted from the generated image feature sequence.

[0072] As an optional embodiment, extracting the sub-region identification sequence from the region identification sequence includes determining at least one region identification with an importance higher than a target threshold in the region identification sequence as the sub-region identification sequence, wherein the importance can be used to represent the importance degree of the corresponding region identification to the pathological result.

[0073] In this embodiment, the importance of each region identification in the region identification sequence is calculated, and the importance of each region identification is compared with a preset target threshold. When there is at least one region identification with an importance higher than the target threshold, the region identification can be determined as the sub-region identification sequence.

[0074] Optionally, the at least one region identification can also be the top K markers selected after sorting each region identification in the region identification sequence according to the importance.

[0075] As an optional embodiment, determining at least one region identification with an importance higher than a target threshold in the region identification sequence as the sub-region identification sequence includes selecting at least one region identification with an importance higher than the target threshold from the region identification sequence based on a lesion site selection module, wherein the lesion site selection module is at least used to determine the importance.

[0076] In this embodiment, the region identification sequence is fed to the lesion site selection module to select at least one region identification with an importance higher than a preset target threshold from the region identification sequence, wherein the lesion site selection module can be at least used to determine the importance of each region identification in the region identification sequence, and the target threshold can be a preset target importance threshold.

[0077] Optionally, the lesion site selection module can be used to learn more unique features and provide visual evidence for lesion positioning by using lesion space information.

[0078] As an optional embodiment, generating the region identification sequence of the skin image includes dividing the skin image into a plurality of region identifications based on a visual transformer model to obtain the region identification sequence, wherein the visual transformer model is trained based on a self-attention mechanism.

[0079] In this embodiment, the skin image can be divided into a plurality of region labels by a visual transformer model, and the plurality of region labels can constitute a region label sequence, wherein the visual transformer model can be trained based on a self-attention mechanism, and can be used for end-to-end training and inference, and the training based on the self-attention mechanism can be training based on a multi-head self-attention module.

[0080] Optionally, dividing the skin image into a plurality of region labels can be based on grid processing of the skin image to generate a plurality of region labels.

[0081] As an optional embodiment, based on the skin lesion features of the skin lesion region and the object type of the to-be-detected object, the skin lesion attribute of the to-be-detected object is determined, comprising: fusing the skin lesion features of a plurality of sub-skin lesion regions corresponding to the sub-region label sequence and the object type to obtain the skin lesion attribute.

[0082] In this embodiment, the skin lesion features of the plurality of sub-skin lesion regions corresponding to the region label sequence and the object type are fused, and based on the fusion result, the skin lesion attribute is determined, wherein the skin lesion features of the plurality of sub-skin lesion regions and the object type can be fused by a context fusion module and a local fusion module, and the context fusion module and the local fusion module can also be used to learn the interaction relationship between the region label sequence and the selected sub-region label sequence.

[0083] As an optional embodiment, the method further comprises: calibrating the skin lesion attribute based on the target skin lesion attribute of the to-be-detected object, and / or calibrating the pathological result based on the target pathological result of the to-be-detected object, so that the skin lesion attribute matches the pathological result.

[0084] In this embodiment, the determined skin lesion attribute of the to-be-detected object is calibrated by determining the target skin lesion attribute of the to-be-detected object, and / or the determined pathological result of the to-be-detected object is calibrated by determining the target pathological result of the to-be-detected object, so as to achieve the purpose of matching the skin lesion attribute with the pathological result.

[0085] Optionally, the target skin lesion attribute of the to-be-detected object can be a skin lesion attribute calculated in advance based on prior knowledge, and the target pathological result of the to-be-detected object can be a disease calculated in advance based on prior knowledge, and calibrating the skin lesion attribute and the pathological result can be realized by a bilateral prediction distillation module.

[0086] In the embodiment of the present application, based on the visual transformer model, the skin image is divided into multiple region labels to obtain a region label sequence, the region label sequence is converted into an image feature sequence of the skin image, a sub-image feature sequence corresponding to the lesion region is determined from the image feature sequence, a sub-region label sequence corresponding to the sub-image feature sequence is obtained, and the lesion features and object types of multiple sub-lesion regions corresponding to the sub-region label sequence are fused to obtain the lesion attribute. The lesion attribute is calibrated to match the pathological result. That is, based on the visual transformer model, the region label sequence is obtained, and the sub-region label sequence is extracted therefrom. The data information corresponding to the sub-region label sequence is fused to achieve accurate matching of the lesion attribute and the pathological result, thereby realizing the technical effect of improving the detection precision of the skin image of the object. The image detection method provided by the embodiment of the present application solves the technical problem of low detection precision of the skin image of the object.

[0087] The embodiment of the present application also provides another skin detection method of an object from the human-computer interaction side.

[0088] Figure 4 is a flowchart of another skin detection method of an object according to the embodiment of the present application. As shown in Figure 4 , the method can include the following steps:

[0089] Step S402, in response to an image input instruction acting on the operation interface, displaying a skin image from a medical diagnosis platform on the operation interface, which is overlaid on the outer surface of the object to be detected.

[0090] In the technical solution provided in the above step S402 of the present application, the image input instruction acts on the operation interface, and the operation interface displays the skin image from the medical diagnosis platform on the operation interface, which is overlaid on the outer surface of the object to be detected, in response to the instruction.

[0091] In this embodiment, the image input instruction can be used to input the skin image data on the outer surface of the object to be detected. For example, by issuing an instruction to input the skin image of the patient's face on the operation interface, the skin image of the patient's face is input in response to the instruction.

[0092] Step S404, in response to a detection operation instruction acting on the operation interface, displaying a pathological result of the object to be detected on the operation interface, wherein the pathological result is obtained by matching lesion data recorded in a lesion database based on a lesion attribute of the object to be detected, the lesion attribute is determined based on lesion features of a lesion region and an object type of the object to be detected, and the lesion region is identified from the skin image.

[0093] In the technical scheme provided in the step S404 of the present application, the detection operation instruction is used to operate the operation interface, and the operation interface displays the pathological result of the to-be-detected object in response to the instruction, wherein the pathological result can be a disease diagnosis of a skin lesion area of the to-be-detected object obtained by matching the pathological data recorded in the lesion database based on the skin lesion attribute of the to-be-detected object, the skin lesion attribute can be determined based on the skin lesion feature of the skin lesion area and the object type of the to-be-detected object, and the skin lesion area can be an image area with the skin lesion feature identified from the skin image of the to-be-detected object.

[0094] In this embodiment, the detection operation instruction can be used to output the pathological result data of the to-be-detected object, for example, by issuing an instruction on the operation interface to output the pathological result data of the to-be-detected object, and the pathological result of the to-be-detected object is output in response to the instruction.

[0095] Optionally, the pathological result of the to-be-detected object can be used to generate a case report to provide more reliable and more interpretable diagnostic results for doctors.

[0096] In the embodiment of the present application, based on the image input instruction and the detection operation instruction acting on the operation interface, the skin image from the medical diagnosis platform and the pathological result of the to-be-detected object are displayed on the operation interface, which provides an interpretable evidence for the pathological result and achieves the purpose of improving the confidence of the diagnostic result, thereby realizing the technical effect of improving the detection accuracy of the skin image of the object, and further, the image detection method provided by the embodiment of the present application solves the technical problem of low detection accuracy of the skin image of the object.

[0097] The embodiment of the present application also provides another skin detection method of an object from the application scenario side.

[0098] Figure 5 is a flowchart of another skin detection method of an object according to the embodiment of the present application. As shown in Figure 5 , the method can include the following steps:

[0099] Step S502: displaying a skin image overlaid on the outer surface of a to-be-detected object on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device.

[0100] In the technical scheme provided in the step S502 of the present application, the skin image overlaid on the outer surface of the to-be-detected object is obtained, and the skin image is displayed on the presentation screen of the VR device or the AR device.

[0101] Step S504: identifying the skin image and determining a skin lesion area of the to-be-detected object, wherein the skin lesion area is an image area with a skin lesion feature in the skin image sensed by the VR device or the AR device.

[0102] In the technical solution provided in the above step S504 of the present application, the skin lesion area in the obtained skin image is determined based on the VR device or the AR device, which can be determining an image area in the skin image that has a skin lesion feature.

[0103] In step S506, the skin lesion attribute of the to-be-detected object is determined based on the skin lesion feature of the skin lesion area and the object type of the to-be-detected object, wherein the skin lesion attribute of the to-be-detected object can be the skin lesion generated by the to-be-detected object.

[0104] In the technical solution provided in the above step S506 of the present application, the skin lesion feature of the skin lesion area is obtained based on the determined skin lesion area, and the object type of the to-be-detected object is obtained based on the to-be-detected object, and then the skin lesion attribute of the to-be-detected object is determined according to the obtained skin lesion feature of the skin lesion area and the object type of the to-be-detected object, wherein the skin lesion attribute can be used to describe the skin lesion generated by the to-be-detected object.

[0105] Optionally, obtaining the skin lesion feature of the skin lesion area based on the determined skin lesion area can be image processing on the determined skin lesion area, so as to obtain the image feature corresponding to the skin lesion area.

[0106] Optionally, obtaining the object type of the to-be-detected object can be based on deep learning to analyze the skin image of the to-be-detected object, or can be based on the data information input by the user in advance.

[0107] In step S508, the pathological result of the to-be-detected object is determined by matching the skin lesion attribute with the lesion data recorded in the lesion database.

[0108] In the technical solution provided in the above step S508 of the present application, the determined skin lesion attribute is matched with the lesion data recorded in the lesion database, and the pathological result of the to-be-detected object is determined based on the obtained matching result.

[0109] Optionally, matching the skin lesion attribute with the lesion data recorded in the lesion database can be based on a posterior distribution calculation method, and the matching result can be a loss function calculated.

[0110] In step S510, the VR device or the AR device is driven to display the skin lesion attribute and the pathological result.

[0111] In the technical solution provided in the above step S510 of the present application, the VR device or the AR device is driven to display the determined skin lesion attribute and the pathological result through the VR device or the AR device.

[0112] Optionally, driving the VR device or the AR device can be sending a driving signal to the VR device or the AR device.

[0113] For example, when the skin lesion attribute and the pathological result are determined, the driving signal can be sent by the user end or the driving signal can be sent by the doctor end, and the display interface of the VR device or the AR device displays the determined skin lesion attribute and the pathological result in response to the driving signal.

[0114] In the embodiment of the present application, by recognizing the skin image displayed on the presentation screen of the VR device or the AR device, the skin lesion area of the to-be-detected object is first determined, then the skin lesion attribute of the to-be-detected object is determined, the pathological result of the to-be-detected object is obtained through the matching result of the skin lesion attribute and the lesion data, and finally the VR device or the AR device is driven to display the skin lesion attribute and the pathological result, thereby realizing the technical effect of improving the detection precision of the skin image of the object. The image detection method provided by the embodiment of the present application solves the technical problem of low detection precision of the skin image of the object.

[0115] The embodiment of the present application also provides another skin detection method of an object from the interaction side.

[0116] Figure 6 is a flowchart of another skin detection method of an object according to the embodiment of the present application. As shown in Figure 6 the method can include the following steps:

[0117] In step S602, a skin image overlaid on the outer surface of the to-be-detected object is obtained by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is the skin image.

[0118] In the technical solution provided in the above step S602 of the present application, the first interface can be an interface for data interaction between a server and a client. The client can input the skin image information into the first interface as a first parameter of the first interface, so as to achieve the purpose of obtaining the skin image information.

[0119] In step S604, the skin image is recognized to determine the skin lesion area of the to-be-detected object, wherein the skin lesion area is an image area in the skin image that has a skin lesion feature.

[0120] In step S606, based on the skin lesion feature of the skin lesion area and the object type of the to-be-detected object, the skin lesion attribute of the to-be-detected object is determined, wherein the skin lesion attribute is used to describe the skin lesion generated by the to-be-detected object.

[0121] In step S608, based on the skin lesion attribute, the lesion data recorded in the lesion database is matched to determine the pathological result of the to-be-detected object.

[0122] In step S610, the second interface is called to output the skin lesion attribute and the pathological result, wherein the second interface includes a second parameter, and the value of the second parameter is the skin lesion attribute and the pathological result.

[0123] In the technical solution provided in the step S610 of the present application, the second interface can be an interface for data interaction between the server and the client, and the server can call the second interface to make the terminal device output the skin lesion attribute and the pathological result in sequence as a parameter of the second interface, so as to achieve the purpose of providing evidence for the explainability of the pathological result.

[0124] In the embodiment of the present application, the skin image covering the outer surface of the to-be-detected object is acquired by calling the first interface; the skin lesion area of the to-be-detected object is determined by recognizing the skin image; the skin lesion attribute of the to-be-detected object is determined based on the skin lesion feature of the skin lesion area and the object type of the to-be-detected object; the pathological result of the to-be-detected object is determined by matching the skin lesion attribute with the lesion data recorded in the lesion database; and the skin lesion attribute and the pathological result are output by calling the second interface. That is, the present application extracts the skin lesion area, matches the skin lesion attribute of the determined skin lesion area with the lesion data recorded in the lesion database to determine the attribute feature of the disease, so as to accurately determine the pathological result, thereby achieving the technical effect of improving the detection precision of the skin image of the object, and further, the image detection method provided in the embodiment of the present application solves the technical problem of low detection precision of the skin image of the object.

[0125] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0126] From the above description of the embodiments, those skilled in the art can clearly understand that the skin detection method of the object according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions to make a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the method described in each embodiment of the present application.

[0127] Embodiment 2

[0128] The preferred implementation of the above method of the embodiment is further described below, which is specifically described by taking a skin detection method of an object as an example.

[0129] In the related art, in order to alleviate the pressure of skin disease diagnosis, an auxiliary diagnosis system is usually designed based on artificial intelligence technology to help doctors diagnose efficiently. The auxiliary system mainly focuses on learning the classification prediction of skin diseases and directly outputting the diagnosis results of the diseases.

[0130] In the related art, a common skin disease auxiliary diagnosis platform is provided, which is an online consultation platform for user terminals (to consumer, referred to as to C). The user himself shoots and uploads the skin condition image. The platform gives a diagnosis result and popular science about the diagnosis result for the skin condition image uploaded by the user. However, the platform only outputs a result with the highest possibility, which does not conform to the logic of doctor diagnosis, and thus has the problem of inaccurate algorithm.

[0131] In another related art, an artificial intelligence method for identifying skin is provided, which directly gives a result based on image features. However, the output diagnosis result is single, and no any explainable evidence is given. Therefore, there is the problem of low confidence of diagnosis result.

[0132] In another related art, a skin disease auxiliary diagnosis method is provided, which is based on a convolutional neural network (Convolutional Neural Networks, referred to as CNN). The method extracts the dependency relationship between local features and long-distance context information, uses a class activation map to locate a discriminant region of a skin lesion and extracts a local target candidate region (region of interest, referred to as ROI) from the detected region, combines global context and fusion representation, and feeds twice to a linear classifier head to achieve the purpose of predicting the distribution of diseases. However, the algorithm has the problems of low calculation efficiency and high calculation resource occupation.

[0133] To solve the above problems, the embodiment provides a skin detection method of an object, which is simultaneously oriented to user terminals and business terminals (to business, referred to as to B). The method gives multiple diagnosis results and a confidence ranking of the multiple diagnosis results, which conforms to the diagnosis process of doctor exclusion and diagnosis. In addition, the embodiment also makes a detailed description of skin lesion attributes, increases the confidence of diagnosis results, and further solves the technical problem of improving the accuracy of skin disease diagnosis.

[0134] Figure 7 is a flowchart of a clinical dermatoscope diagnosis method based on a transformer model according to an embodiment of the present application, as shown in Figure 7As shown, the method can include the following steps:

[0135] Step S702, generating an image feature sequence based on the region marker sequence.

[0136] In this embodiment, based on the extracted region marker sequence, a corresponding image feature sequence is generated, wherein the region marker sequence can be obtained by grid processing the skin image or the input image, or the input image based on the visual transformer model, and each local marker has a specific position encoding.

[0137] Optionally, generating a corresponding image feature sequence based on the extracted region marker sequence can be inputting the region marker sequence into a multi-head self-attention module to generate the image feature sequence.

[0138] Step S704, removing redundant features from the extracted image feature sequence.

[0139] In this embodiment, the extracted image feature sequence is input into a lesion site selection module to remove redundant image features in the image feature sequence, wherein the lesion site selection module can remove redundant features by automatically extracting the marker sequence of the lesion area.

[0140] Step S706, fusing global information and significant local information.

[0141] In this embodiment, based on the context fusion module and the local fusion module, the global information and the significant local information are fused, wherein the global information can be the extracted region marker sequence, and the significant local information can be the obtained sub-region marker sequence.

[0142] Optionally, after fusing the global information and the significant local information, it can also include classifying conditions and attributes through two independent self-attention modules.

[0143] Step S708, correcting the misclassified samples.

[0144] In this embodiment, the misclassified samples can be corrected based on the bilateral prediction distillation module by obtaining the co-occurrence matrix of the pre-calculated disease and lesion attribute.

[0145] The embodiment of the present application is based on a visual transformer model, first generates an image feature sequence through a region label sequence, removes redundant features from the extracted image feature sequence, fuses global information and significant local information, and finally corrects the misclassified samples, so as to enhance the learning ability of the lesion features under different distributions, thereby realizing the technical effect of improving the accuracy of skin disease diagnosis, and the image detection method provided by the embodiment of the present application solves the technical problem of low accuracy of skin disease diagnosis caused by the complexity of lesion attributes.

[0146] Figure 8 is a schematic diagram of a general skin disease automatic diagnosis system according to an embodiment of the present application. It should be noted that the general skin disease automatic diagnosis system can be used to execute the above-mentioned clinical dermatoscope diagnosis method based on the transformer model. As shown in Figure 8 , the system can at least include a lesion site selection module 804 and a bilateral prediction distillation module 808:

[0147] The network backbone module (not shown in the figure) is used to divide the input image into a plurality of region labels according to the visual transformer model, denoted as {x n , n∈{1, 2..., N p}}. Wherein,

[0148] N p = (H x W) / P 2 (1)

[0149] In the above formula (1), N p represents the number of region labels, and p represents the side length of the region label.

[0150] The local image block is flattened and gets the feature label of the region label through linear mapping, denoted as x′ n Each local label is also given a learnable position vector to retain the position encoding information, and at the same time, a class label is introduced, denoted as which can be used to record the global feature information.

[0151] The input image region label sequence is represented as {t n ∈R D , n∈{0, 1,..., N p}}, D represents the length, and the encoder of the visual transformer model is composed of L consecutive layers of multi-head self-attention modules. The output of each layer is denoted as t l , l∈{1,..., L}.

[0152] Due to the great variation in the distribution and scale of skin lesions in clinical dermatology images, the model needs to have the ability to locate the lesion area to describe its attributes and accurately associate it with the disease category. However, the label sequence generated by the visual transformer model is spatially redundant for the task performed, so a lesion site selection module, also known as a lesion label selection module, is designed to guide the encoder of the visual transformer to select the localized labels related to the lesion at different levels. The lesion site selection module 804 is used to first calculate the attention matrix for each attention head in all multi-head self-attention modules:

[0153]

[0154] The normalized exponential function (softmax) is calculated for the first row and the first column of the matrix A m respectively:

[0155] and

[0156] The above formulas (2) and (3) can represent the attention scores between class labels and other spatial labels. The mutual attention scores are calculated for all heads, and finally the importance of local labels is ranked and the top K labels are selected from the l-th layer, where N h represents the number of attention heads, Q (Query) and K (Key) represent the inputs of the attention module.

[0157] Since determining the disease type not only requires fine-grained local features, but also requires global context information such as the body part where the lesion appears, the interaction between the class labels and the selected local labels needs to be learned, that is, the fusion of global and local semantic information. The disease attribute joint learning module (not shown in the figure) uses the selected local labels and the class labels of the last transformer layer to fuse the features.

[0158] First, two additional fusion modules are introduced, and two classification heads are defined for a given sample, and the outputs of the two additional fusion modules introduced are mapped to and where the two classification heads defined can essentially be two fully connected layers (FC), and the two additional fusion modules introduced can be a context fusion module and a local fusion module, n c and n a represent the number of condition and attribute classes respectively.

[0159] Secondly, a joint optimization objective function is defined:

[0160]

[0161] In the above formula (4), N c denotes the number of sample number annotations of diseases, N a denotes the number of sample number annotations of attributes, and β denotes the inverse of the effective sample number of classes, denotes the actual sample number, denotes the label of disease type single-hot encoding, denotes the multi-hot lesion attribute label, and l c / l a denotes the indicator function of the condition / attribute, and when the condition / attribute label is available, l c / l a is 1, otherwise l c / l a is 0.

[0162] Finally, the class imbalance problem is handled based on the class balanced loss.

[0163] In addition to the head supervision training of the disease type and the lesion attribute, the prior knowledge of the dermatologist can be used to calibrate the system when a misjudgment occurs. The bilateral prediction distillation module 808 calculates the posterior distribution of the disease given the lesion attribute based on the statistics of the disease and the attribute:

[0164]

[0165] The posterior distribution of the attribute is calculated given the disease diagnosis:

[0166]

[0167] Each entry of the two matrices can be calculated as follows:

[0168]

[0169] In the above formula (7), 1[·] denotes the indicator function, and k denotes the sample index.

[0170] For a given sample k, the two simultaneous occurrence probabilities and are calculated, and the relative entropy (Kullback-Leibler divergence, referred to as KL divergence) between the two probabilities is obtained, wherein the KL divergence can be used to represent the non-symmetry measure of the difference between two probability distributions.

[0171] Since different diseases may share the same attribute, the distribution of is multi-modal, while the actual diagnosis gives a unique most likely result, that is, p cThe distribution of the skin lesions can be unimodal. Thus, and p c The exact match between p

[0172] Two consistency measures are defined for attributes and disease classes respectively, which penalize only when p

[0173]

[0174]

[0175]

[0176] The resulting loss function is composed of three parts:

[0177] L total = L joint + a1L cond_attr + a2L attr_cond (11)

[0178] In the embodiment of the application, the skin image obtained is processed by four different modules to extract the identification sequence of the skin lesion area in the skin image. After the global information and the significant local information are fused, the loss function of the attribute and the disease class is obtained by calculating the posterior distribution, so as to accurately determine the pathological result, thereby realizing the technical effect of improving the detection precision of the skin image. Furthermore, the image detection method provided by the embodiment of the application solves the technical problem of low detection precision of the skin image caused by the complexity of the skin lesion attribute.

[0179] Through the above description of the embodiments, those skilled in the art can clearly understand that the skin detection method of the object according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the method of each embodiment of the application.

[0180] Embodiment 3

[0181] According to the embodiment of the application, a skin detection device for an object is also provided, which is used to implement the skin detection method of the object as shown in the above Figure 3

[0182] Figure 9 ​is a schematic diagram of a skin detection device of an object according to an embodiment of the present application, as shown in Figure 9 The skin detection device 900 of the object can include an acquisition unit 902, a first determination unit 904, a second determination unit 906, and a first matching unit 908.

[0183] The acquisition unit 902 is configured to acquire a skin image covering an outer surface of a to-be-detected object.

[0184] The first determination unit 904 is configured to identify the skin image and determine a lesion area of the to-be-detected object, wherein the lesion area is an image area in the skin image where a lesion feature exists.

[0185] The second determination unit 906 is configured to determine a lesion attribute of the to-be-detected object based on the lesion feature of the lesion area and an object type of the to-be-detected object, wherein the lesion attribute is used to describe a lesion generated by the to-be-detected object.

[0186] The first matching unit 908 is configured to match lesion data recorded in a lesion database based on the lesion attribute, and determine a pathological result of the to-be-detected object.

[0187] It should be noted that the above-mentioned first acquisition unit 902, first determination unit 904, second determination unit 906, and first matching unit 908 correspond to steps S302 to S308 in Embodiment 1, and the four units have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in the above-mentioned embodiment one. It should be noted that the above-mentioned units as part of the device can run in the AR / VR device provided in Embodiment 1.

[0188] According to an embodiment of the present application, a skin detection device of an object for implementing the above-mentioned Figure 4 skin detection method of the object is also provided.

[0189] Figure 10 is a schematic diagram of a skin detection device of an object according to an embodiment of the present application, as shown in Figure 10 The skin detection device 1000 of the object can include a first display unit 1002 and a second display unit 1004.

[0190] The first display unit 1002 is configured to display a skin image covering an outer surface of a to-be-detected object from a medical diagnosis platform on an operation interface in response to an image input instruction acting on the operation interface.

[0191] The second display unit 1004 is configured to display a pathological result of the to-be-detected object on the operation interface in response to a detection operation instruction acting on the operation interface, wherein the pathological result is obtained by matching lesion data recorded in a lesion database based on a skin lesion attribute of the to-be-detected object, and the skin lesion attribute is determined based on a skin lesion feature of a skin lesion area and an object type of the to-be-detected object, and the skin lesion area is obtained by recognizing a skin image.

[0192] It should be noted that the first display unit 1002 and the second display unit 1004 correspond to steps S402 to S404 in Embodiment 1, and the two units have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above units can be run in the AR / VR device provided in Embodiment 1 as part of the device.

[0193] According to the embodiments of the present application, a skin detection device for implementing the skin detection method of the object is also provided. Figure 5

[0194] Figure 11 FIG. 1 is a schematic diagram of a skin detection device of an object according to an embodiment of the present application, as shown in the figure, the skin detection device 1100 of the object can include a display unit 1102, a third determination unit 1104, a fourth determination unit 1106, a second matching unit 1108, and a driving unit 1110. Figure 11

[0195] The display unit 1102 is configured to display a skin image overlaid on an outer surface of a to-be-detected object on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device.

[0196] The third determination unit 1104 is configured to recognize the skin image and determine a skin lesion area of the to-be-detected object, wherein the skin lesion area is an image area in which a skin lesion feature exists in the skin image sensed by the VR device or the AR device.

[0197] The fourth determination unit 1106 is configured to determine a skin lesion attribute of the to-be-detected object based on a skin lesion feature of the skin lesion area and an object type of the to-be-detected object, wherein the skin lesion attribute is used to describe a skin lesion generated by the to-be-detected object.

[0198] The second matching unit 1108 is configured to match lesion data recorded in a lesion database based on the skin lesion attribute, and determine a pathological result of the to-be-detected object.

[0199] The driving unit 1110 is configured to drive the VR device or the AR device to display the skin lesion attribute and the pathological result.

[0200] ​​It should be noted that the above display unit 1102, third determination unit 1104, fourth determination unit 1106, second matching unit 1108 and driving unit 1110 correspond to steps S502 to S510 in Embodiment 1, and the five units have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above units can be run in the AR / VR device provided in Embodiment 1 as part of the device.

[0201] According to an embodiment of the present application, a skin detection device for implementing the above-mentioned skin detection method of an object is also provided. Figure 6 The skin detection device of the object shown in the skin detection method of the object.

[0202] Figure 12 is a schematic diagram of a skin detection device of an object according to an embodiment of the present application, as shown in the skin detection device of the object 1200 can include: a first calling unit 1202, a fifth determination unit 1204, a sixth determination unit 1206, a third matching unit 1208 and a second calling unit 1210. Figure 12

[0203] The first calling unit 1202 is configured to obtain a skin image overlaid on the outer surface of the object to be detected by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is the skin image.

[0204] The fifth determination unit 1204 is configured to identify the skin image and determine the skin lesion area of the object to be detected, wherein the skin lesion area is an image area in the skin image that has a skin lesion feature.

[0205] The sixth determination unit 1206 is configured to determine the skin lesion attribute of the object to be detected based on the skin lesion feature of the skin lesion area and the object type of the object to be detected, wherein the skin lesion attribute is used to describe the skin lesion generated by the object to be detected.

[0206] The third matching unit 1208 is configured to match the skin lesion attribute with the lesion data recorded in the lesion database to determine the pathological result of the object to be detected.

[0207] The second calling unit 1210 is configured to output the skin lesion attribute and the pathological result by calling a second interface, wherein the second interface includes a second parameter, and the value of the second parameter is the skin lesion attribute and the pathological result.

[0208] ​It should be noted that the first calling unit 1202, the fifth determining unit 1204, the sixth determining unit 1206, the third matching unit 1208 and the second calling unit 1210 correspond to steps S602 to S610 in Embodiment 1, and the five units have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above units can run in the AR / VR device provided in Embodiment 1 as part of the device.

[0209] In this embodiment, by determining the lesion area, the lesion attribute of the lesion area is matched with the lesion data recorded in the lesion database, the attribute characteristics of the disease are determined, and the lesion attribute and the pathological result are displayed on the related interface, so as to accurately determine the pathological result, thereby realizing the technical effect of improving the detection accuracy of the skin image of the object, and further, the image detection method provided in the embodiment of the application solves the technical problem of low detection accuracy of the skin image of the object.

[0210] Embodiment 4

[0211] The embodiment of the application can provide a computer terminal, which can be any one of the computer terminal devices in the computer terminal group. Alternatively, in this embodiment, the computer terminal can be replaced by a terminal device such as a mobile terminal.

[0212] Alternatively, in this embodiment, the computer terminal can be located in at least one of the network devices of the computer network.

[0213] In this embodiment, the computer terminal can execute the program code of the following steps in the skin detection method of the application object: obtaining a skin image covering the outer surface of the object to be detected; identifying the skin image to determine the lesion area of the object to be detected, wherein the lesion area is an image area in the skin image that has a skin lesion feature; based on the skin lesion feature of the lesion area and the object type of the object to be detected, determining the skin lesion attribute of the object to be detected, wherein the skin lesion attribute is used to describe the skin lesion generated by the object to be detected; based on the skin lesion attribute, matching the lesion data recorded in the lesion database to determine the pathological result of the object to be detected.

[0214] Alternatively, Figure 13 is a structural block diagram of a computer terminal according to an embodiment of the application. As shown in Figure 13 The computer terminal A can include one or more (only one is shown in the figure) processors 1302, a memory 1304, and a transmission device 1306.

[0215] The memory can be configured to store software programs and modules, such as program instructions / modules corresponding to the skin detection method and device of the object in the embodiments of the present application. The processor executes various functions and data processing by running the software programs and modules stored in the memory, i.e., implements the skin detection method of the object as described above. The memory can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the computer terminal A through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0216] The processor can call information and application programs stored in the memory through the transmission device to perform the following steps: acquiring a skin image covering the outer surface of the object to be detected; identifying the skin image to determine a lesion area of the object to be detected, wherein the lesion area is an image area in the skin image that has a lesion feature; determining a lesion attribute of the object to be detected based on the lesion feature of the lesion area and the object type of the object to be detected, wherein the lesion attribute is used to describe the lesion generated by the object to be detected; and based on the lesion attribute, matching the lesion data recorded in the lesion database to determine the pathological result of the object to be detected.

[0217] Optionally, the processor can further execute program codes of the following steps: identifying the skin image to obtain a region identification sequence, wherein each region identification in the region identification sequence is used to represent a skin region of the object to be detected; and extracting a sub-region identification sequence from the region identification sequence, wherein each region identification in the sub-region identification sequence is used to represent a sub-lesion region in the lesion area.

[0218] Optionally, the processor can further execute program codes of the following steps: converting the region identification sequence into an image feature sequence of the skin image, wherein an image feature in the image feature sequence is used to represent a skin region corresponding to a region identification in the region identification sequence; determining a sub-image feature sequence corresponding to the lesion area from the image feature sequence, wherein the lesion area includes a skin region corresponding to an image feature in the sub-image feature sequence; and determining a sub-region identification sequence corresponding to the sub-image feature sequence.

[0219] Optionally, the processor can further execute program codes of the following steps: determining at least one region identification in the region identification sequence with an importance higher than a target threshold value as the sub-region identification sequence, wherein the importance can be used to represent the importance degree of the corresponding region identification to the pathological result.

[0220] Optionally, the processor can further execute program codes of the following steps: selecting at least one region identifier with importance higher than a target threshold from the sequence of region identifiers based on a lesion site selection module, wherein the lesion site selection module is used at least for determining the importance.

[0221] Optionally, the processor can further execute program codes of the following steps: dividing the skin image into a plurality of region identifiers based on the visual transformer model to obtain the sequence of region identifiers, wherein the visual transformer model is trained based on a self-attention mechanism.

[0222] Optionally, the processor can further execute program codes of the following steps: fusing the lesion features and the object type of the plurality of sub-lesion regions corresponding to the sequence of sub-region identifiers to obtain the lesion attribute.

[0223] Optionally, the processor can further execute program codes of the following steps: calibrating the lesion attribute based on a target lesion attribute of the object to be detected, and / or calibrating the pathological result based on a target pathological result of the object to be detected, so as to match the lesion attribute with the pathological result.

[0224] As an optional example, the processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: in response to an image input instruction acting on the operation interface, displaying a skin image from a medical diagnosis platform on the operation interface, which is overlaid on the outer surface of the object to be detected; and in response to a detection operation instruction acting on the operation interface, displaying a pathological result of the object to be detected on the operation interface, wherein the pathological result is obtained by matching lesion data recorded in a lesion database based on a lesion attribute of the object to be detected, the lesion attribute is determined based on lesion features of a lesion region and an object type of the object to be detected, and the lesion region is identified from the skin image.

[0225] As an optional example, the processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: displaying a skin image overlaid on the outer surface of the object to be detected on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device; identifying the skin image to determine a lesion region of the object to be detected, wherein the lesion region is an image region of the skin image sensed by the VR device or the AR device, in which lesion features exist; determining a lesion attribute of the object to be detected based on the lesion features of the lesion region and an object type of the object to be detected, wherein the lesion attribute is used to describe a lesion generated by the object to be detected; determining a pathological result of the object to be detected by matching lesion data recorded in a lesion database based on the lesion attribute; and driving the VR device or the AR device to display the lesion attribute and the pathological result.

[0226] As an optional example, the processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtaining a skin image covering the outer surface of the to-be-detected object through calling a first interface, wherein the first interface comprises a first parameter, and the parameter value of the first parameter is the skin image; identifying the skin image to determine a lesion area of the to-be-detected object, wherein the lesion area is an image area in the skin image where a lesion feature exists; determining a lesion attribute of the to-be-detected object based on the lesion feature of the lesion area and the object type of the to-be-detected object, wherein the lesion attribute is used to describe the lesion generated by the to-be-detected object; matching the lesion attribute with lesion data recorded in a lesion database to determine a pathological result of the to-be-detected object; and outputting the lesion attribute and the pathological result through calling a second interface, wherein the second interface comprises a second parameter, and the value of the second parameter is the lesion attribute and the pathological result.

[0227] The embodiment of the present application provides a skin detection method for an object, the lesion attribute of the determined lesion area is matched with lesion data recorded in a lesion database to determine the attribute feature of the disease, the pathological result is accurately determined, the precision of the skin image detection for the object is improved, and the technical problem of low precision of the skin image detection for the object is solved.

[0228] Those skilled in the art can understand that Figure 13 The structure shown is only schematic, and the computer terminal can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, or the like. Figure 13 The structure of the above-mentioned electronic device is not limited. For example, the computer terminal A can further comprise more or fewer components (such as a network interface, a display device, etc.) than Figure 13 shown, or have a different configuration from Figure 13 shown.

[0229] Those skilled in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer readable storage medium, and the storage medium can include a flash disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, etc.

[0230] Embodiment 5

[0231] The embodiment of the present application further provides a computer readable storage medium. Optionally, in the embodiment, the computer readable storage medium can be used to save the program code executed by the skin detection method of the object provided in the first embodiment.

[0232] Optionally, in the embodiment, the computer readable storage medium can be located in any computer terminal in the computer terminal group in the computer network, or in any mobile terminal in the mobile terminal group.

[0233] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: obtaining a skin image covering the outer surface of the object to be detected; identifying the skin image to determine a lesion area of the object to be detected, wherein the lesion area is an image area in the skin image with a lesion feature; determining a lesion attribute of the object to be detected based on the lesion feature of the lesion area and the object type of the object to be detected, wherein the lesion attribute is used to describe the lesion generated by the object to be detected; and based on the lesion attribute, matching the lesion data recorded in the lesion database to determine the pathological result of the object to be detected.

[0234] Optionally, the computer readable storage medium can further execute program code for the following steps: identifying the skin image to obtain a region identification sequence, wherein each region identification in the region identification sequence is used to represent a skin area of the object to be detected; and extracting a sub-region identification sequence from the region identification sequence, wherein each region identification in the sub-region identification sequence is used to represent a sub-lesion area in the lesion area.

[0235] Optionally, the computer readable storage medium can further execute program code for the following steps: converting the region identification sequence into an image feature sequence of the skin image, wherein each image feature in the image feature sequence is used to represent the skin area corresponding to the region identification in the region identification sequence; determining a sub-image feature sequence corresponding to the lesion area from the image feature sequence, wherein the lesion area includes the skin area corresponding to the image feature in the sub-image feature sequence; and determining a sub-region identification sequence corresponding to the sub-image feature sequence.

[0236] Optionally, the computer readable storage medium can further execute program code for the following steps: determining at least one region identification in the region identification sequence with an importance higher than a target threshold as the sub-region identification sequence, wherein the importance can be used to represent the importance degree of the corresponding region identification to the pathological result.

[0237] Optionally, the computer readable storage medium can further execute program code for the following steps: selecting at least one region identification with an importance higher than a target threshold from the region identification sequence based on a lesion site selection module, wherein the lesion site selection module is used to determine at least the importance.

[0238] Optionally, the computer readable storage medium can further execute program codes of the following steps: dividing the skin image into a plurality of region labels based on the visual transformer model to obtain a sequence of region labels, wherein the visual transformer model is trained based on a self-attention mechanism.

[0239] Optionally, the computer readable storage medium can further execute program codes of the following steps: fusing the lesion features and the object type of the plurality of sub-lesion regions corresponding to the sequence of sub-region labels to obtain the lesion attribute.

[0240] Optionally, the computer readable storage medium can further execute program codes of the following steps: calibrating the lesion attribute based on the target lesion attribute of the to-be-detected object, and / or calibrating the pathological result based on the target pathological result of the to-be-detected object, so as to match the lesion attribute with the pathological result.

[0241] As an optional example, the computer readable storage medium is configured to store program codes for executing the following steps: in response to an image input instruction acting on the operation interface, displaying a skin image from a medical diagnosis platform on the operation interface, the skin image being overlaid on the outer surface of the to-be-detected object; in response to a detection operation instruction acting on the operation interface, displaying a pathological result of the to-be-detected object on the operation interface, wherein the pathological result is obtained by matching lesion data recorded in a lesion database based on a lesion attribute of the to-be-detected object, the lesion attribute is determined based on lesion features of a lesion region and an object type of the to-be-detected object, and the lesion region is identified from the skin image.

[0242] As an optional example, the computer readable storage medium is configured to store program codes for executing the following steps: displaying a skin image overlaid on the outer surface of the to-be-detected object on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device; identifying the skin image to determine a lesion region of the to-be-detected object, wherein the lesion region is an image region of the skin image sensed by the VR device or the AR device, in which lesion features exist; determining a lesion attribute of the to-be-detected object based on the lesion features of the lesion region and an object type of the to-be-detected object, wherein the lesion attribute is used to describe a lesion generated by the to-be-detected object; determining a pathological result of the to-be-detected object by matching lesion data recorded in a lesion database based on the lesion attribute; and driving the VR device or the AR device to display the lesion attribute and the pathological result.

[0243] As an optional example, the computer readable storage medium is configured to store program code for performing the following steps: obtaining a skin image covering an outer surface of the to-be-detected object by calling a first interface, wherein the first interface comprises a first parameter, and a parameter value of the first parameter is the skin image; identifying the skin image to determine a lesion area of the to-be-detected object, wherein the lesion area is an image area in the skin image where a lesion feature exists; determining a lesion attribute of the to-be-detected object based on the lesion feature of the lesion area and an object type of the to-be-detected object, wherein the lesion attribute is used to describe a lesion generated by the to-be-detected object; matching the lesion attribute with lesion data recorded in a lesion database to determine a pathological result of the to-be-detected object; and outputting the lesion attribute and the pathological result by calling a second interface, wherein the second interface comprises a second parameter, and a value of the second parameter is the lesion attribute and the pathological result.

[0244] The above-mentioned embodiment numbers of the application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0245] In the above-mentioned embodiments of the application, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0246] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other ways. Among them, the above-mentioned device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or modules shown or discussed can be indirect coupling or communication connection through some interfaces, units or modules, which can be electrical or other forms.

[0247] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0248] In addition, each functional unit in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0249] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0250] The above is only the preferred embodiment of the present application, it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, can make a number of improvements and refinements, these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A skin detection method of a subject, characterized by, The method comprises the following steps: acquiring a skin image covering an outer surface of a to-be-detected object; identifying the skin image to determine a lesion area of the to-be-detected object, wherein the lesion area is an image area in the skin image in which a lesion feature exists, and the identification of the skin image comprises obtaining a region identification sequence, wherein each region identification in the region identification sequence represents a skin area of the to-be-detected object; and extracting a sub-region identification sequence from the region identification sequence, wherein each region identification in the sub-region identification sequence represents a sub-lesion area in the lesion area; determining a lesion attribute of the to-be-detected object based on a lesion feature of the lesion area and an object type of the to-be-detected object, wherein the lesion attribute is used to describe a lesion generated by the to-be-detected object; matching the lesion attribute with lesion data recorded in a lesion database to determine a pathological result of the to-be-detected object, wherein the pathological result is used to represent a result of a disease diagnosis on the lesion area; wherein the determination of the lesion attribute of the to-be-detected object based on the lesion feature of the lesion area and the object type of the to-be-detected object comprises: fusing the lesion feature and the object type through a context fusion module and a local fusion module to obtain the lesion attribute, wherein the lesion feature comprises a lesion feature of the lesion area in a skin area and lesion features of a plurality of sub-lesion areas in the lesion area, the context fusion module is a module for fusing the region identification sequence, and the local fusion module is a module for fusing the sub-region identification sequence.

2. The method of claim 1, wherein, The extraction of the sub-region identification sequence from the region identification sequence comprises: converting the region identification sequence into an image feature sequence of the skin image, wherein an image feature in the image feature sequence represents the skin area corresponding to the region identification in the region identification sequence; determining a sub-image feature sequence corresponding to the sub-region identification sequence from the image feature sequence, wherein the lesion area comprises the skin area corresponding to the image feature in the sub-image feature sequence; and determining the sub-region identification sequence corresponding to the sub-image feature sequence.

3. The method of claim 1, wherein, The extraction of the sub-region identification sequence from the region identification sequence comprises: determining at least one region identification in the region identification sequence with an importance higher than a target threshold as the sub-region identification sequence, wherein the importance represents the importance degree of the corresponding region identification to the pathological result.

4. The method of claim 3, wherein, The determination of the at least one region identification in the region identification sequence with the importance higher than the target threshold as the sub-region identification sequence comprises: selecting the at least one region identification with the importance higher than the target threshold from the region identification sequence based on a lesion site selection module, wherein the lesion site selection module is used to determine at least the importance.

5. The method of claim 1, wherein, The generation of the region identification sequence from the skin image comprises: The skin image is divided into a plurality of region labels based on a visual transformer model, to obtain a region label sequence, wherein the visual transformer model is obtained based on self-attention mechanism training.

6. The method of claim 1, wherein, The lesion feature and the object type are fused through a context fusion module and a local fusion module to obtain the lesion attribute, including: The lesion feature and the object type of a plurality of sub-lesion regions corresponding to the sub-region label sequence are fused through the context fusion module and the local fusion module to obtain the lesion attribute.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: The lesion attribute is calibrated based on a target lesion attribute of the to-be-detected object, and / or the pathological result is calibrated based on a target pathological result of the to-be-detected object, so that the lesion attribute matches the pathological result.

8. A skin detection method of a subject, characterized by, It includes: In response to an image input instruction acting on the operation interface, a skin image from a medical diagnosis platform is displayed on the operation interface, which is overlaid on the outer surface of the to-be-detected object; In response to a detection operation instruction acting on the operation interface, a pathological result of the to-be-detected object is displayed on the operation interface, wherein the pathological result is used to represent a result obtained by diagnosing the lesion area, and the pathological result is obtained by matching lesion data recorded in a lesion database based on a lesion attribute of the to-be-detected object, the lesion attribute is obtained by fusing a lesion feature of the lesion area and an object type of the to-be-detected object through a context fusion module and a local fusion module, the lesion feature includes a lesion feature of the lesion area in the skin area and a lesion feature of a plurality of sub-lesion areas in the lesion area, each region label in a sub-region label sequence is used to represent a sub-lesion area in the lesion area, the sub-region label sequence is obtained by identifying the skin image and extracting from a region label sequence, each region label in the region label sequence is used to represent a skin area of the to-be-detected object, the context fusion module is a module for fusing the region label sequence, and the local fusion module is a module for fusing the sub-region label sequence.

9. A skin detection method of a subject, characterized by, It includes: A skin image overlaid on the outer surface of the to-be-detected object is displayed on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device; The skin image is identified to determine a lesion area of the to-be-detected object, wherein the lesion area is an image area in the skin image sensed by the VR device or the AR device that has a lesion feature, including: identifying the skin image to obtain a region label sequence, wherein each region label in the region label sequence is used to represent a skin area of the to-be-detected object; and extracting a sub-region label sequence from the region label sequence, wherein each region label in the sub-region label sequence is used to represent a sub-lesion area in the lesion area; Based on the skin lesion characteristics of the lesion area and the object type of the object to be detected, the skin lesion attributes of the object to be detected are determined, wherein the skin lesion attributes are used to describe the skin lesions produced by the object to be detected; Based on the skin lesion attributes, the pathological results of the subject to be tested are determined by matching them with the lesion data recorded in the lesion database. The pathological results are used to represent the results obtained by diagnosing the condition of the skin lesion area. Drive the VR device or the AR device to display the skin lesion attributes and the pathological results; The method for determining the skin lesion attributes of the target object based on the skin lesion features of the lesion region and the object type of the target object includes: fusing the skin lesion features and the object type through a contextual fusion module and a local fusion module to obtain the skin lesion attributes. The skin lesion features include: the skin lesion features of the lesion region in the skin region, and the skin lesion features of multiple sub-lesion regions in the lesion region. The contextual fusion module is a module for fusing the region identifier sequence, and the local fusion module is a module for fusing the sub-region identifier sequence.

10. A skin detection method of a subject, characterized by, include: A skin image covering the outer surface of the object to be detected is obtained by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is the skin image; Identifying the skin image and determining the lesion region of the target object, wherein the lesion region is an image region in the skin image that has lesion features, includes: identifying the skin image and obtaining a region identifier sequence, wherein each region identifier in the region identifier sequence is used to represent a skin region of the target object; extracting a sub-region identifier sequence from the region identifier sequence, wherein each region identifier in the sub-region identifier sequence is used to represent a sub-lesion region within the lesion region; Based on the skin lesion characteristics of the lesion area and the object type of the object to be detected, the skin lesion attributes of the object to be detected are determined, wherein the skin lesion attributes are used to describe the skin lesions produced by the object to be detected; Based on the skin lesion attributes, the pathological results of the subject to be tested are determined by matching them with the lesion data recorded in the lesion database. The pathological results are used to represent the results obtained by diagnosing the condition of the skin lesion area. The skin lesion attributes and pathological results are output by calling the second interface, wherein the second interface includes a second parameter, the value of which is the skin lesion attributes and the pathological results; The skin lesion attribute of the to-be-detected object is determined based on a skin lesion feature of the skin lesion area and an object type of the to-be-detected object, and the determination includes: fusing the skin lesion feature and the object type through a context fusion module and a local fusion module to obtain the skin lesion attribute, wherein the skin lesion feature includes a skin lesion feature of the skin lesion area in the skin area and skin lesion features of a plurality of sub-skin lesion areas in the skin lesion area, the context fusion module is a module for fusing the region identification sequence, and the local fusion module is a module for fusing the sub-region identification sequence.

11. A computer readable storage medium, characterized in that, The computer readable storage medium comprises a stored program, wherein the program, when executed by a processor, controls a device in which the computer readable storage medium is located to perform the method of any one of claims 1 to 10.

12. A processor, comprising: The processor is configured to execute a program, wherein the program, when executed, performs the method of any one of claims 1 to 10.

Images