Information detection method and device

Abstract

The invention discloses an information detection method and device, and belongs to the technical field of electronic equipment. The method is executed by an electronic device, N light-emitting light sources are integrated in the electronic device, one light-emitting light source is used for emitting light of one wave band, and the method comprises the steps that N skin spectral images are obtained, one skin spectral image is collected when one light-emitting light source emits light, and the N skin spectral images all comprise objects to be detected; and detecting the to-be-detected object according to the N skin spectrum images to obtain a target classification result of the to-be-detected object.

Description

Technical Field

[0001] This application belongs to the field of information detection technology, specifically relating to an information detection method and apparatus. Background Technology

[0002] Currently, professional medical equipment and doctors are used to manually interpret images of the subject for testing to obtain the results. With the development of dermatopathology testing technology, timely detection and diagnosis of moles, as one of the common skin lesions, are crucial for preventing skin cancer. Traditional mole detection methods require professional medical equipment and doctors to manually interpret captured skin images, which is costly and inefficient. With the widespread use of wearable devices, using everyday worn devices for mole detection offers convenience and practicality.

[0003] In recent years, with the widespread use of wearable devices and smartphones, it has become possible to achieve medical-grade monitoring using everyday devices such as watches and mobile phones. Currently, some studies have attempted to determine the benign or malignant nature of moles by analyzing their appearance through mobile phone photos; however, these methods suffer from low accuracy in mole detection. Summary of the Invention

[0004] The purpose of this application is to provide an information detection method and apparatus that can achieve accurate detection of moles through a portable electronic device.

[0005] In a first aspect, embodiments of this application provide an information detection method, executed by an electronic device, wherein the electronic device integrates N light-emitting sources, each light-emitting source emitting light in a specific wavelength band, and the method includes: N skin spectral images are acquired, each skin spectral image is acquired when light is emitted from a light source, and each of the N skin spectral images contains the object to be detected; Based on the N skin spectral images, the object to be detected is detected to obtain the target classification result of the object to be detected.

[0006] Secondly, embodiments of this application provide an information detection device applied to an electronic device, wherein the electronic device integrates N light-emitting light sources, each light-emitting light source emitting light in a specific wavelength band, and the device includes: The acquisition module is used to acquire N skin spectral images. Each skin spectral image is acquired when a light source emits light. Each of the N skin spectral images contains the object to be detected. The detection module is used to detect the object to be detected based on the N skin spectral images and obtain the target classification result of the object to be detected.

[0007] Thirdly, embodiments of this application provide an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can run on the processor, and the programs or instructions, when executed by the processor, implement the method as described in the first aspect.

[0008] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the method described in the first aspect.

[0009] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0010] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0011] The solution in this application embodiment can be applied to an electronic device integrating N light sources. Each light source emits light in one wavelength band. By detecting the object to be detected based on the skin spectral image acquired when each light source emits light, the target classification result of the object to be detected can be obtained. Thus, the image acquired in this application embodiment is a multispectral image. Since each skin spectral image is a reflected light image of the object to be detected acquired in a different wavelength band, it helps to reveal the characteristics of the object to be detected in different tissue layers. Therefore, when detecting the object to be detected based on the multispectral image, it can be detected from the characteristics of the object to be detected contained in different tissue layers, rather than relying solely on the appearance boundary of the object to be detected. Thus, the solution in this application embodiment improves the detection accuracy of the object to be detected. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating the information detection method provided in some embodiments of this application; Figure 2 This is a flowchart illustrating the information detection method provided in some embodiments of this application; Figure 3 This is a schematic diagram of the architecture of an information detection system provided in some embodiments of this application; Figure 4 These are schematic diagrams illustrating the structure of an information detection device according to some embodiments of this application; Figure 5 These are schematic diagrams illustrating the structure of an electronic device according to some embodiments of this application; Figure 6These are schematic diagrams illustrating the hardware structure of an electronic device according to some embodiments of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0014] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and are not limited in number; for example, a first object can be one or N objects. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0015] The relevant technologies involved in the embodiments of the present invention will be explained below.

[0016] In existing technologies, when detecting moles using electronic devices, a flashlight source is used to illuminate the skin in the area where the mole is located, and the benign or malignant nature of the mole is determined by analyzing the appearance of the skin image. However, because the flashlight source in existing technologies is a single-band light source, it can only illuminate the surface layer of the skin. Therefore, the benign or malignant nature of the mole can only be determined based on its appearance in the surface skin image, resulting in low detection accuracy.

[0017] The technical solution of this application embodiment can be applied to scenarios where portable electronic devices are used to detect moles on a user's skin. For example, a user looks in the mirror and notices that a mole on their face seems to have grown larger than before. The user wants to use their mobile phone to check the benign or malignant nature of the mole, as well as the trend of the mole's change compared to before, i.e., whether the mole has grown larger or worsened. This can guide the user on whether they need to seek medical attention and receive professional treatment.

[0018] The information detection method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0019] Figure 1This is a flowchart illustrating an information detection method provided in an embodiment of this application. The subject executing the information detection method can be an electronic device, such as, but not limited to, a personal computer (PC), smartphone, tablet computer, personal digital assistant (PDA), or extended reality (XR) device. Here, the XR device can be, but not limited to, a virtual reality (VR) device or an augmented reality (AR) device.

[0020] The electronic devices described in this application embodiment are portable electronic devices, such as mobile phones.

[0021] It should be noted that the electronic device in this embodiment may integrate N light-emitting sources, each of which emits light in one wavelength band. Here, N is a positive integer, and N≥2.

[0022] The aforementioned N light sources may include, for example, light sources that emit R-band light, light sources that emit G-band light, light sources that emit B-band light, and light sources that emit near-infrared (NIR) band light.

[0023] like Figure 1 As shown, the information detection method provided in this application embodiment may include steps 110-120.

[0024] Step 110: Obtain N skin spectral images.

[0025] The N skin spectral images can be skin spectral images acquired when N light sources emit light sequentially, meaning each skin spectral image is acquired when one light source emits light. For example, an electronic device can control N light sources to emit light sequentially in a certain order. For instance, the N light sources can emit light in the order of emitting R-band light, G-band light, B-band light, and NIR-band light, and then acquire a skin spectral image when emitting R-band light, another when emitting G-band light, another when emitting B-band light, and another when emitting NIR-band light, thus obtaining 4 skin spectral images.

[0026] In some embodiments of this application, skin spectral images can be acquired using a camera in an electronic device, and each skin spectral image contains an object to be detected. The object to be detected can be any object to be examined, such as a mole on the user's face in the example above.

[0027] In the following examples, the object to be detected is a mole on the user's face as described in the above example.

[0028] It should be noted that the N light sources in the electronic device can each emit light individually or in combination. That is, one of the N skin spectral images mentioned above can be acquired when one light source emits light, or when multiple light sources emit light simultaneously. This embodiment uses the example of acquiring a skin spectral image when one light source emits light as an example.

[0029] Step 120: Based on N skin spectral images, detect the object to be detected and obtain the target classification result of the object to be detected.

[0030] The target classification result can be the classification result of the object to be detected. In the case that the object to be detected is a mole on the user's face in the example above, the target classification result can be whether the mole is benign or malignant.

[0031] In some embodiments of this application, in order to improve the detection accuracy of the object to be detected, step 120 may specifically include: Register N skin spectral images to obtain N registered skin spectral images; The registered N skin spectral images are fused to obtain a fused skin spectral image; The target object is obtained by segmenting the object to be detected in the fused skin spectral image; The target objects are classified to obtain the target classification results of the target objects to be detected.

[0032] Among them, the fused skin spectral image can be a single skin spectral image obtained by fusing N registered skin spectral images.

[0033] The target detection object can be the detection object obtained by segmenting the object to be detected in the fused skin spectral image. In the example above, the target detection object is the mole on the user's face, except that the target detection object is segmented from the fused skin spectral image obtained by fusing N skin spectral images.

[0034] In some embodiments of this application, the target detection object is different from the object to be detected, and the target detection object better reflects its morphological structure in various tissue layers of the skin. This is because moles are located in various tissue layers of the skin. When a mole is illuminated by a light source emitting light of a certain wavelength, the resulting spectral image can only observe the structure of the mole in the tissue layer corresponding to the light source in that skin spectral image. For example, R-band light can only illuminate the surface layer of the skin. Therefore, in the skin spectral image acquired when emitting R-band light, only the morphological structure of the mole in the surface layer of the skin can be observed, and the morphological structure of the mole in the dermis and subcutaneous tissue cannot be observed. After fusing N skin spectral images, the mole exists in each tissue layer in the resulting fused skin spectral image. Thus, the target detection object segmented from this fused skin spectral image reflects its morphological structure in the skin better than the object to be detected in the N skin spectral images.

[0035] In some embodiments of this application, N skin spectral images can be registered to obtain N registered skin spectral images. Then, the N registered skin spectral images can be fused to obtain a fused skin spectral image. The target object to be detected in the fused skin spectral image can be segmented to obtain the target detection object. Then, the target detection object can be classified to obtain the target classification result of the target object to be detected.

[0036] In the embodiments of this application, N skin spectral images are registered to obtain N registered skin spectral images. Then, the N registered skin spectral images are fused to obtain a fused skin spectral image. The target object is extracted from the fused skin spectral image to obtain the target detection object. In this way, the target detection object exists in each skin tissue layer, which helps to reveal the characteristics of the target detection object in different tissue layers. Thus, when detecting the target object based on this multispectral image, the detection can be performed from the characteristics of the target detection object contained in different tissue layers, rather than relying solely on the appearance boundary of a single target object, thereby improving the detection accuracy of the target object.

[0037] In some embodiments of this application, when the N light sources include light sources emitting R-band light, G-band light, B-band light, and near-infrared (NIR) light, the N skin spectral images may include R-band skin spectral images, G-band skin spectral images, B-band skin spectral images, and NIR skin spectral images. Here, the R-band skin spectral image is an image acquired when the electronic device emits R-band light, the G-band skin spectral image is an image acquired when the electronic device emits G-band light, the B-band skin spectral image is an image acquired when the electronic device emits B-band light, and the NIR skin spectral image is an image acquired when the electronic device emits NIR-band light.

[0038] To further improve the detection accuracy of the target object, the registration of N skin spectral images to obtain N registered skin spectral images may specifically include: Identify reference skin spectral images in R-ray skin spectral images, G-ray skin spectral images, B-ray skin spectral images, and NIR-ray skin spectral images; The pixels in the reference skin spectral image are matched with the pixels in each skin spectral image to be matched, resulting in the transformation matrix between the reference skin spectral image and each skin spectral image to be matched. The target transformation matrix is ​​determined based on the transformation matrix between the reference skin spectral image and each skin spectral image to be matched. Based on the target transformation matrix, each skin spectral image to be matched is transformed into the coordinate system corresponding to the reference skin spectral image, resulting in N registered skin spectral images.

[0039] The reference skin spectral image can be one of the R-light skin spectral images, G-light skin spectral images, B-light skin spectral images, and NIR-light skin spectral images used as a reference during registration. The reference skin spectral image can be any one of the R-light skin spectral images, G-light skin spectral images, B-light skin spectral images, and NIR-light skin spectral images. For example, the reference skin spectral image can be an R-light skin spectral image.

[0040] The skin spectral images to be matched can be skin spectral images other than the reference skin spectral image from among N skin spectral images. For example, if the reference skin spectral image is an R-light skin spectral image, the skin spectral images to be matched can be a G-light skin spectral image, a B-light skin spectral image, and a NIR-light skin spectral image.

[0041] The target transformation matrix can be a transformation matrix obtained from the transformation matrix between the reference skin spectral image and each skin spectral image to be matched.

[0042] In some embodiments of this application, slight angle changes or displacement errors caused by hand tremors may occur during the capture of R-ray skin spectral images, G-ray skin spectral images, B-ray skin spectral images, and NIR-ray skin spectral images. Therefore, an image registration method is needed to align all skin spectral images to a unified reference image coordinate system. Thus, a reference skin spectral image needs to be selected, for example, an R-ray skin spectral image.

[0043] Then, the pixels in the reference skin spectral image are matched with the pixels in each skin spectral image to be matched, and the transformation matrix between the reference skin spectral image and each skin spectral image to be matched is obtained. Then, based on the transformation matrix between the reference skin spectral image and each skin spectral image to be matched, the target transformation matrix is ​​obtained. Then, based on the target transformation matrix, each skin spectral image to be matched can be transformed into the coordinate system corresponding to the reference skin spectral image, and N registered skin spectral images are obtained.

[0044] In the embodiments of this application, by unifying N skin spectral images to the same reference image coordinate system, the slight angle changes or displacement errors caused by hand tremors during the shooting process of the N skin spectral images can be resolved, thereby avoiding the impact of errors caused by slight angle errors or hand tremors on the detection accuracy of the object to be detected. The solution of this application improves the detection accuracy of the object to be detected.

[0045] In some embodiments of this application, to improve the efficiency of obtaining the target transformation matrix and the robustness of registration, the step of matching pixels in the reference skin spectral image with pixels in each skin spectral image to be matched, to obtain the transformation matrix between the reference skin spectral image and each skin spectral image to be matched, may specifically include: Extract at least one first keypoint from the pixels of the reference skin spectral image; For each skin spectral image to be matched, find a second keypoint in the skin spectral image that matches each first keypoint; Based on at least one first key point and at least one second key point in each skin spectral image to be matched, the transformation matrix between the reference skin spectral image and each skin spectral image to be matched is obtained respectively. The step of determining the target transformation matrix based on the transformation matrix between the reference skin spectral image and each skin spectral image to be matched may specifically include: From the transformation matrices between the reference skin spectral image and each skin spectral image to be matched, select the transformation matrix that matches the pixels of each skin spectral image to be matched most frequently with the pixels in the reference skin spectral image, and use this matrix as the target transformation matrix.

[0046] At least one of the first key points can be pixels extracted from each pixel of the reference skin spectral image that can characterize the features of the reference skin spectral image. Specifically, this can involve constructing the scale space of the reference skin spectral image and then finding DoG extrema as the first key points.

[0047] For each skin spectral image to be matched, at least one second keypoint can be a keypoint in the skin spectral image to be matched that matches at least one first keypoint.

[0048] In some embodiments of this application, at least one first key point can be extracted from the pixels of a reference skin spectral image. Then, for each skin spectral image to be matched, a second key point matching each first key point is found in the skin spectral image to be matched. Thus, for each skin spectral image to be matched, a transformation matrix between the reference skin spectral image and the skin spectral image to be matched can be obtained based on at least one second key point and at least one first key point from the pixels of the reference skin spectral image. That is, each skin spectral image to be matched has its corresponding transformation matrix.

[0049] Then, for each skin spectral image to be matched, the transformation matrix that matches the pixels of each skin spectral image to be matched most frequently with the pixels in the reference skin spectral image can be selected and used as the target transformation matrix.

[0050] In the embodiments of this application, the target transformation matrix is ​​determined by referencing a first key point in the skin spectral image and a second key point in each skin spectral image to be matched, rather than by referencing all pixels in the skin spectral image and all pixels in each skin spectral image to be matched. This improves the efficiency of obtaining the target transformation matrix. Furthermore, when determining the target transformation matrix, the transformation matrix that maximizes the match between pixels in each skin spectral image and pixels in the reference skin spectral image is selected as the target transformation matrix. This ensures the alignment quality of most matchable regions in the image, effectively resisting interference from local differences and outliers, and improving the robustness of registration.

[0051] In some embodiments of this application, in order to obtain a clearer, less noisy, and more information-rich fused skin spectral image, the process of fusing the registered N skin spectral images to obtain the fused skin spectral image may specifically include: The registered N skin spectral images are stitched together along the channel dimension to obtain a stitched image; Based on the weight of each channel in the N channels, the image features of each channel in the stitched image are weighted and calculated to obtain the fused skin spectral image.

[0052] The stitched image can be an N-channel spectral image obtained by stitching together N registered skin spectral images in the channel dimension.

[0053] In some embodiments of this application, the registered N skin spectral images are stitched together along the channel dimension to obtain a stitched image. Then, the image features of each channel in the stitched image can be weighted according to the weight of each channel in the N channels to obtain a fused skin spectral image.

[0054] In some embodiments of this application, the registered N skin spectral images are stitched together along the channel dimension to obtain a stitched image. Then, based on the weight of each channel in the N channels, the image features of each channel in the stitched image are weighted and calculated to obtain a fused skin spectral image. This fusion image can be obtained based on a pre-trained fusion model. The fusion model can be a pre-trained deep learning-based neural network model, support vector machine model, decision tree model, etc.

[0055] To enhance the ability of the fusion model to identify information from different spectral channels, a spectral attention module can be introduced into the fusion model. This module can perform global average pooling on each channel in the spatial dimension to obtain a channel description vector. This channel description vector is then input into a fully connected network to learn the channel weights, thus obtaining the weights of each of the N channels.

[0056] In the embodiments of this application, since different spectral channels carry different biological tissue information and are affected by noise to varying degrees, some channels may have clear signals in some areas and high noise in others. Therefore, by stitching together the registered N skin spectral images along the channel dimension, a stitched image can be obtained. Then, based on the weight of each channel in the N channels, the image features of each channel in the stitched image are weighted and calculated to obtain a fused skin spectral image. In this way, by giving greater weight to channels with high signal-to-noise ratio and rich information, while suppressing channels with high noise, redundant information, or invalid information, the final fused image can integrate the useful information of all channels, thereby obtaining a fused skin spectral image that is clearer, less noisy, and richer in information than any single channel.

[0057] In existing technologies, when segmenting the detection object in an image, the image is directly input into a segmentation model to directly segment the detection object in the image. However, this method has low segmentation accuracy. To solve the above problem, in some embodiments of this application, the segmentation of the object to be detected in the fused skin spectral image to obtain the target detection object may specifically include: Skin color regions are extracted from the fused skin spectral image to obtain the mask image corresponding to the fused skin spectral image; Based on the mask image, the target object is segmented in the fused skin spectral image to obtain the target detection object.

[0058] The mask image can be obtained by extracting the skin color region from the fused skin spectral image. It can be understood that this mask image is obtained by extracting the foreground skin color region from the fused skin spectral image, thus eliminating the background region in the fused skin spectral image and avoiding the influence of the background region on the detection of the object to be detected.

[0059] In some embodiments of this application, the fused skin spectral image can be segmented using a thresholding method, i.e., selecting one or more grayscale values ​​as "thresholds". Pixels with grayscale values ​​higher than the threshold are classified as foreground (white), and those lower than the threshold are classified as background (black), and vice versa. Since the fused skin spectral image may have a large difference in grayscale values ​​between the skin region and the background region in a specific channel, a global threshold can be used to obtain the mask image. Alternatively, an edge detection algorithm can be used to detect points (i.e., edges) with drastic grayscale changes in the fused skin spectral image, then these edges can be connected to form a closed contour, and finally the interior of the contour can be filled to obtain the mask. A region growing method can also be used to obtain the mask image. Alternatively, the fused skin spectral image can be input into a mask model, and the mask image can be obtained based on the mask model; this is not limited here.

[0060] After obtaining the mask image corresponding to the fused skin spectral image, the target object in the fused skin spectral image can be segmented based on the mask image to obtain the target detection object.

[0061] In the embodiments of this application, by extracting the skin color region in the fused skin spectral image, a mask image corresponding to the fused skin spectral image is obtained. Based on the mask image, the object to be detected in the fused skin spectral image is segmented, which can improve the segmentation accuracy of the object to be detected.

[0062] In some embodiments of this application, in order to improve the accuracy of target object extraction and thus improve the accuracy of target object classification, the step of segmenting the target object in the fused skin spectral image based on the mask image to obtain the target object may specifically include: Based on the mask image, the object to be detected in the fused skin spectral image is segmented to obtain the candidate region of the object to be detected; The connected components in the candidate regions of the detection objects are filtered to obtain the target detection objects.

[0063] Among them, the candidate region of the detection object can be the candidate region of the target detection object, that is, the target detection object is included in the region, but the target detection object extracted in the region is not accurate enough.

[0064] In some embodiments of this application, the object to be detected in the fused skin spectral image can be segmented based on the mask image to obtain the candidate region of the object to be detected. The target object extracted from the candidate region of the object to be detected is not accurate enough. Therefore, the connected components in the candidate region of the object to be detected can be filtered to obtain the target object to be detected.

[0065] In the embodiments of this application, the target object is obtained by segmenting the fused skin spectral image based on the mask image. Then, the connected components in the target object candidate region are filtered to obtain the target object. Instead of directly using the target object in the target object candidate region as the final target object, the accuracy of target object extraction is improved, thereby improving the accuracy of target object classification results.

[0066] In some embodiments of this application, in order to improve the accuracy of target detection object extraction, the step of filtering connected components in the candidate region of the detection object to obtain the target detection object may specifically include: When there is only one connected component, the object detected in the smallest bounding rectangle of the connected component is taken as the target detection object. When there are at least two connected components, the detection object in the smallest bounding rectangle of the target connected component is taken as the target detection object.

[0067] The target connected component can be one of at least two connected components. The specific selection of the target connected component will be described in detail in subsequent embodiments.

[0068] In some embodiments of this application, when there is only one connected component in the candidate region of the detected object, the smallest bounding rectangle of that connected component can be used as the extraction region for the mole, and the detected object can be extracted from that region. This detected object is the target detected object. In other words, the detected object in the smallest bounding rectangle of the connected component is the target detected object.

[0069] When there are at least two connected components in the candidate region of the target object, one connected component can be selected as the target connected component. This target connected component is the main extraction region of the mole, and the target object is extracted from this region. In other words, the target object is the object within the smallest bounding rectangle of the target connected component.

[0070] In the embodiments of this application, the main extraction region of the mole is determined based on the number of connected components in the candidate region of the detection object, and the target detection object is extracted based on the main extraction region, instead of directly extracting the target detection object from the entire candidate region of the detection object. This reduces the extraction region of the target detection object, reduces the amount of computation, and improves the extraction accuracy of the target detection object.

[0071] In some embodiments of this application, to further improve the accuracy of target detection object extraction, when the number of connected components is at least two, the above-mentioned method may further include: Determine the score for each connected component based on its attribute information; The connected component with the highest score among all connected components is identified as the target connected component.

[0072] The attribute information of a connected component can include at least its area, aspect ratio, roundness, and color density. The roundness of the connected component can be obtained based on the following formula (1):

[0073] In the above formula (1), For this connected component area, , here Let be the coordinates of the pixels in the connected component. Let be the perimeter of the connected region.

[0074] The color density of the connected components mentioned above can be obtained based on the following formula (2):

[0075] In the above formula (2), For the pixels in this connected component The grayscale value.

[0076] In some embodiments of this application, the score of each connected component can be determined based on the attribute information of each connected component. Specifically, for each connected component, the score of the connected component can be obtained by weighted summation of its area, aspect ratio, roundness, and color density.

[0077] Then, select the connected component with the highest score from the scores of each connected component and determine it as the target connected component.

[0078] In the embodiments of this application, when there are at least two connected regions, the connected region with the highest score is selected based on the area, aspect ratio, roundness and color density of each connected region, and this is taken as the target connected region. In this way, the area of ​​the target detection object contained in the target connected region is the largest and the color density is the largest, which can better extract the target detection object and improve the extraction accuracy of the target detection object.

[0079] In some embodiments of this application, in order to improve the efficiency and accuracy of object segmentation and the robustness of the image segmentation model, the segmentation of the object to be detected in the fused skin spectral image based on the mask image to obtain the target detection object may specifically include: The mask image and the fused skin spectral image are input into the image segmentation model to segment the object to be detected in the fused skin spectral image, thereby obtaining the target detection object.

[0080] The image segmentation model can be a pre-trained model used to segment the objects to be detected in the fused skin spectral image.

[0081] The image segmentation model described above can be a U-Net network model, which consists of two parts: an encoder for extracting high-level semantic features from the fused skin spectral image and mask image, a decoder for progressively restoring the spatial resolution of the fused skin spectral image and mask image and performing pixel-level prediction, and a skip connection layer that directly passes the features of each layer of the encoder to the corresponding decoder layer to preserve boundary detail information.

[0082] To address the common issue of uneven skin tone in mole images during capture, the U-Net network model can be supplemented with a Convolutional Block Attention Module (CBAM) and a Squeeze-and-Excitation attention (SE) module. The CBAM module learns regions of interest across both channel and spatial attention dimensions, enhancing sensitivity to mole edges and texture variations. The SE module adaptively enhances useful features along the channel dimension while suppressing irrelevant background. These attention mechanisms help the model focus more on uneven color regions and hair-occluded areas in the image, reducing false segmentation.

[0083] The above image segmentation model is obtained by training a segmentation model to be trained. The structure of the segmentation model to be trained is the same as that of the image segmentation model. The loss function of the segmentation model to be trained can be a binary cross loss function and a Dice loss function. That is to say, the loss function of the image segmentation model is a binary cross loss function and a Dice loss function. The specific binary cross loss function can be shown in the following formula (3):

[0084] In the above formula (3), The detected objects are predicted based on the segmentation model to be trained. The input to the segmentation model to be trained consists of the labels of the detected objects, i.e., the real detected objects labeled in the sample fused skin spectral image. For pixels in the image, i.e. For each pixel in the detected object predicted based on the segmentation model to be trained, For each pixel in the real detection object labeled in the skin spectral image of the sample fusion, This represents the total number of pixels in the fused skin spectral image of the sample.

[0085] The aforementioned sample fused skin spectral image can be input into the segmentation model to be trained to extract the fused skin spectral image of the detected object. The acquisition method of this sample fused skin spectral image is the same as that of the aforementioned fused skin spectral image, and will not be repeated here.

[0086] The Dice loss function can be represented by the following formula (4):

[0087] The characters in formula (4) above have the same meaning as those in formula (3). This is a pre-set minimum value, which is to avoid It is 0.

[0088] The loss function of the image segmentation model can be expressed as follows (5):

[0089] In the above formula (5) and This is a weighting coefficient, the value of which can be set by the user according to their needs, and is not limited in this embodiment.

[0090] In the embodiments of this application, an image segmentation model is used to segment the objects to be detected in the fused skin spectral image, rather than through manual annotation and extraction, thereby improving the efficiency and accuracy of object segmentation. Furthermore, the image segmentation model employs a composite loss function, enhancing its robustness.

[0091] In some embodiments of this application, after obtaining the target detection object, classifying the target detection object and determining its benign or malignant nature can be based on a trained lightweight neural network, such as the RetNet neural network. This RetNet model integrates the ResNet residual structure, multi-scale receptive fields, and attention mechanisms to improve the ability to identify malignant features. The multi-scale convolutional layers in this RetNet model can extract texture and color difference features from regions of different sizes, effectively handling variations in the size and blurriness of moles. Residual connection layers use residual connections to enhance the stability and trainability of deep networks. The attention mechanism layer can weight and strengthen regions in the image related to the shape and color changes of moles, filtering out interfering backgrounds.

[0092] In some embodiments of this application, when obtaining the target classification result of the object to be detected, the method described above may further include: Determine the confidence level of the target classification result; Based on the target classification results and confidence level, the risk level of the object to be tested is determined.

[0093] In some embodiments of this application, after obtaining the target classification result of the object to be detected, the confidence level of the target classification result can also be obtained. Then, the risk level of the object to be detected can be determined based on the target classification result and the confidence level. Here, the risk level is high risk, medium risk, and low risk.

[0094] Based on the target classification results and confidence level, the risk level of the object to be tested can be determined according to the rules shown in Table 1 below.

[0095] Table 1

[0096] In some embodiments of this application, after obtaining the target classification result of the object to be detected, the method described above may further include: Obtain the target classification results of the objects to be detected at different time periods; Based on the target classification results of the objects to be detected at different time periods, the changing trend of the objects to be detected is obtained; Based on the changing trends, determine the risk score of the object to be tested.

[0097] In some embodiments of this application, after obtaining the target classification result of the object to be detected, the target classification results of the object to be detected at different time periods can be time-series aligned to obtain the changing trend of the object to be detected. Based on the changing trend, the risk score of the object to be detected can be determined. Specifically, a Long Short-Term Memory (LSTM) network model can be constructed, and the target classification results of the object to be detected at different time periods and the fused skin spectral images at different time periods can be input into the LSTM model. Based on the LSTM model, the changing trend of the object to be detected is analyzed, and based on the changing trend of the object to be detected, the risk score of the object to be detected is determined. This risk score can be used to indicate the probability of malignancy of the object to be detected, and then the user can use the risk score to see whether they need to seek medical treatment.

[0098] In one example, if a user detects moles on their face once a month, that is, obtains the target classification result of the mole once a month, the user can store the fused skin spectrum image obtained from each image captured over time, resulting in the image sequence shown in the following formula (6):

[0099] In the above formula (6), This refers to the fused skin spectral image obtained from a single test. For example, if a user underwent four tests over a period of four months, from January 1, 2025 to April 1, 2025 (one test per month), with the first test on January 1, 2025, the second on February 1, 2025, the third on March 1, 2025, and the fourth on April 1, 2025, then... The image shows the fused skin spectral image obtained during the first detection on January 1, 2025. This is the fused skin spectral image obtained during the second detection on February 1, 2025. This is the fused skin spectral image obtained during the third detection on March 1, 2025. This is the fused skin spectral image obtained during the fourth detection on April 1, 2025.

[0100] When there are N light sources, N spectral images will be captured each time. The features of different spectral images will be extracted to obtain the multidimensional feature vectors obtained for each capture. That is, for a single capture, the multidimensional feature vectors obtained are as shown in the following formula (7).

[0101]

[0102] In the above formula (7), During a single shoot, The spectral image captured at a given time, for example, taking a light source with 4 light sources, is the fused skin spectral image obtained during the first detection at 2025.1.1 in the above formula (6). A total of four spectral images were captured during this shooting session. , here From respectively Features extracted from R-band light skin spectral images acquired at specific times during R-band light emission, from... Features extracted from G-band skin spectral images acquired while G-band light is emitted at specific times, from... Features extracted from B-band light skin spectral images acquired at specific times of B-band light emission, from... Features extracted from NIR light spectral images of skin acquired while emitting NIR band light at various times.

[0103] The multidimensional feature vectors obtained from each shot are sorted according to the time series and then input into the LSTM model to obtain the risk score of the object to be detected.

[0104] In the embodiments of this application, by analyzing the changing trends of the target classification results of the object to be detected at different time periods, a risk score of the object to be detected is obtained. This allows for the analysis of the changing trends of the object to be detected at different time periods, enabling dynamic prediction of the development trajectory of the object to be detected, and assisting in judging the risk progress of the object to be detected, so that users can take corresponding measures in a timely manner.

[0105] In some embodiments of this application, in order to improve the detection accuracy of the object to be detected, after acquiring N skin spectral images, the method described above may further include: For each skin spectral image, a contrast enhancement process is performed to obtain an enhanced skin spectral image; The enhanced skin spectral image is filtered to obtain the filtered skin spectral image. Morphological restoration is performed on the filtered skin spectral image to obtain the target skin spectral image; The step of detecting the target object based on N skin spectral images to obtain the target classification result of the target object may specifically include: Based on N target skin spectral images, the target object is detected, and the target classification result of the target object is obtained.

[0106] The target skin spectral image can be an image obtained by morphological restoration of a filtered skin spectral image.

[0107] In some embodiments of this application, the acquired N skin spectral images may be affected by interference factors such as uneven skin tone, complex background, and hair occlusion. Therefore, before detecting the target object based on the N skin spectral images and obtaining the target classification result of the target object, the N skin spectral images can be preprocessed separately. Specifically, the preprocessing can be as follows: for each skin spectral image, perform contrast enhancement processing to obtain an enhanced skin spectral image; then, perform filtering processing on the enhanced skin spectral image to obtain a filtered skin spectral image; and finally, perform morphological restoration on the filtered skin spectral image to obtain the target skin spectral image. Subsequently, the target object can be detected based on the N target skin spectral images to obtain the target classification result of the target object.

[0108] The above-described contrast enhancement process for skin spectral images yields an enhanced skin spectral image. Specifically, this can involve histogram equalization, which involves sequentially scanning each pixel of the skin spectral image, calculating its gray-level histogram, calculating the cumulative distribution function of the gray-level histogram, and then determining the mapping relationship between the input and output based on the cumulative distribution function and histogram equalization principles. Finally, image transformation is performed based on the mapping relationship to obtain the enhanced skin spectral image.

[0109] The above-mentioned filtering process on the enhanced skin spectrum image to obtain the filtered skin spectrum image can be achieved by using an edge detection algorithm to filter the enhanced skin spectrum image. This can effectively detect image structures such as hair and grayscale icons. Specifically, Gaussian filtering can be used to denoise the enhanced skin spectrum image to obtain a denoised skin spectrum image. Then, the gradient magnitude and gradient direction of the denoised skin spectrum image can be calculated. Specifically, the Sobel operator can be used to calculate the gradient magnitude and gradient direction of each pixel in the denoised skin spectrum image. Then, pixel-level maximum suppression processing is performed on the gradient magnitude map to obtain the edge image. Finally, a double threshold processing algorithm is used to classify the edge image to obtain the filtered skin spectrum image.

[0110] The above-mentioned morphological restoration of the filtered skin spectral image to obtain the target skin spectral image can be achieved by eliminating fine hair structures through morphological opening operations and then performing interpolation filling using convolution kernels to obtain the restored image, which is the target skin spectral image.

[0111] In the embodiments of this application, for each skin spectral image, the skin spectral image can be preprocessed to eliminate the influence of interference factors such as uneven skin color, complex background, and hair occlusion, thereby improving the detection accuracy of the object to be detected.

[0112] To better understand the information detection method of this application embodiment, the information detection method of this application embodiment will be described in detail below with specific scenarios.

[0113] like Figure 2 As shown, the information detection method provided in this application embodiment may include steps 210-280.

[0114] Step 210: Obtain N skin spectral images.

[0115] Step 220: For each skin spectral image, preprocess the skin spectral image to obtain the target skin spectral image.

[0116] In step 220, each skin spectral image can be preprocessed to obtain the target skin spectral image. The specific preprocessing steps can be referred to the preprocessing operation of each skin spectral image in the above embodiment, and will not be repeated here.

[0117] Step 230: Register the N target skin spectral images to obtain the registered N target skin spectral images.

[0118] In step 230, the operation of registering N target skin spectral images to obtain N registered target skin spectral images is the same as the process of registering N skin spectral images to obtain N registered skin spectral images in the above embodiment, and will not be described again here.

[0119] Step 240: Fuse the registered N target skin spectral images to obtain a fused skin spectral image.

[0120] In step 240, the process of fusing the registered N target skin spectral images to obtain a fused skin spectral image is the same as the process of fusing the registered N skin spectral images to obtain a fused skin spectral image in the above embodiment, and will not be repeated here.

[0121] Step 250: Segment the object to be detected in the fused skin spectral image to obtain the target detection object.

[0122] Step 260: Classify the target detection object to obtain the target classification result of the target object to be detected.

[0123] Steps 250-260 are consistent with the process described in the above embodiment of segmenting the object to be detected in the fused skin spectral image to obtain the target detection object, classifying the target detection object, and obtaining the target classification result of the target object to be detected. They will not be described again here.

[0124] Step 270: Based on the target classification results of the objects to be detected in different time periods, obtain the changing trend of the objects to be detected.

[0125] Step 280: Determine the risk score of the object to be tested based on the trend of change.

[0126] In steps 270-280, the process is consistent with that in the above embodiment, which obtains the changing trend of the object to be detected based on the target classification results of the object to be detected at different time periods, and determines the risk score of the object to be detected based on the changing trend. It will not be described again here.

[0127] To better understand the information detection method of the embodiments of this application, the system architecture for implementing the information detection method of the embodiments of this application will be described in detail below.

[0128] like Figure 3 As shown, the information detection system provided in this application embodiment may include a multi-head attention mechanism module 310, an image segmentation module 320, a classification model 330, and a long-term risk assessment module 340 for cross-temporal information.

[0129] After acquiring N skin spectral images, the N skin spectral images can be input into the multi-head attention mechanism module 310. Based on the multi-head attention mechanism module 310, the N skin spectral images can be preprocessed to obtain N target skin spectral images. Then, the N target skin spectral images are registered to obtain N registered target skin spectral images. Finally, the N registered target skin spectral images are fused to obtain a fused skin spectral image.

[0130] The fused skin spectral image is input into the image segmentation module 320. The image segmentation module 320 can extract the skin color region in the fused skin spectral image to obtain the mask image corresponding to the fused skin spectral image. Based on the mask image, the object to be detected in the fused skin spectral image is segmented to obtain the target detection object.

[0131] Then, the target detection object is input into the classification model 330, and the classification model 330 is used to classify the target detection object to obtain the target classification result.

[0132] By inputting the target classification results of the objects to be detected at different time periods and the fused skin spectral images at different times into the long-term risk assessment module 340 with cross-time series information, the changing trend of the objects to be detected can be obtained, and the risk score of the objects to be detected can be determined based on the changing trend.

[0133] The information detection method provided in this application can be executed by an information detection device. This application uses an information detection device executing the information detection method as an example to illustrate the information detection device provided in this application.

[0134] Figure 4 This is a schematic diagram illustrating the structure of an information detection device according to an exemplary embodiment. The information detection device is applied to an electronic device, which integrates N light-emitting sources, each emitting light in a specific wavelength band. Figure 4 As shown, the information detection device 400 may include: The acquisition module 410 is used to acquire N skin spectral images. Each skin spectral image is acquired when a light source emits light. Each of the N skin spectral images contains the object to be detected. The detection module 420 is used to detect the object to be detected based on the N skin spectral images and obtain the target classification result of the object to be detected.

[0135] The solution in this application embodiment can be applied to an electronic device integrating N light sources. Each light source emits light in one wavelength band. By detecting the object to be detected based on the skin spectral image acquired when each light source emits light, the target classification result of the object to be detected can be obtained. Thus, the image acquired in this application embodiment is a multispectral image. Since each skin spectral image is a reflected light image of the object to be detected acquired in a different wavelength band, it helps to reveal the characteristics of the object to be detected in different tissue layers. Therefore, when detecting the object to be detected based on the multispectral image, it can be detected from the characteristics of the object to be detected contained in different tissue layers, rather than relying solely on the appearance boundary of the object to be detected. Thus, the solution in this application embodiment improves the detection accuracy of the object to be detected.

[0136] In some embodiments of this application, the detection module may specifically be used for: The N skin spectral images are registered to obtain N registered skin spectral images; The registered N skin spectral images are fused to obtain a fused skin spectral image; The object to be detected in the fused skin spectral image is segmented to obtain the target detection object; The target detection object is classified to obtain the target classification result of the target object to be detected.

[0137] In some embodiments of this application, the N light sources include light sources emitting R-band light, light sources emitting G-band light, light sources emitting B-band light, and light sources emitting near-infrared (NIR) light; and the N skin spectral images include R-band skin spectral images, G-band skin spectral images, B-band skin spectral images, and NIR skin spectral images. The detection module can specifically be used for: A reference skin spectral image is identified from the R-ray skin spectral image, the G-ray skin spectral image, the B-ray skin spectral image, and the NIR-ray skin spectral image; The pixels in the reference skin spectral image are matched with the pixels in each skin spectral image to be matched, thereby obtaining the transformation matrix between the reference skin spectral image and each skin spectral image to be matched. The skin spectral images to be matched are the skin spectral images other than the reference skin spectral image among the N skin spectral images. The target transformation matrix is ​​determined based on the transformation matrix between the reference skin spectral image and each skin spectral image to be matched. Based on the target transformation matrix, each skin spectral image to be matched is transformed into the coordinate system corresponding to the reference skin spectral image to obtain N registered skin spectral images.

[0138] In some embodiments of this application, the detection module may specifically be used for: The registered N skin spectral images are stitched together along the channel dimension to obtain a stitched image, which is an N-channel spectral image. Based on the weight of each channel in the N channels, the image features of each channel in the stitched image are weighted and calculated to obtain a fused skin spectral image.

[0139] In some embodiments of this application, the detection module may specifically be used for: The skin color region in the fused skin spectral image is extracted to obtain the mask image corresponding to the fused skin spectral image; Based on the mask image, the object to be detected in the fused skin spectral image is segmented to obtain the target detection object.

[0140] In some embodiments of this application, the detection module may specifically be used for: Based on the mask image, the object to be detected in the fused skin spectral image is segmented to obtain candidate regions of the object to be detected; The connected components in the candidate region of the detection object are filtered to obtain the target detection object.

[0141] In some embodiments of this application, the detection module may specifically be used for: When there is only one connected component, the object detected in the smallest bounding rectangle of the connected component is taken as the target detection object. When the number of connected components is at least two, the detection object in the smallest bounding rectangle of the target connected component is taken as the target detection object, and the target connected component is one of the at least two connected components.

[0142] In some embodiments of this application, when the number of connected domains is at least two, the apparatus further includes: The first determining module is used to determine the score of each connected component based on the attribute information of each connected component. The attribute information of a connected component includes at least the area, aspect ratio, roundness, and color density of the connected component. The connected component with the highest score among the scores of each connected component is determined as the target connected component.

[0143] In some embodiments of this application, the detection module may specifically be used for: The mask image and the fused skin spectral image are input into the image segmentation model to segment the object to be detected in the fused skin spectral image to obtain the target detection object; The image segmentation model uses a binary crossover loss function and a Dice loss function as its loss functions.

[0144] In some embodiments of this application, the acquisition module is further configured to acquire the target classification results of the object to be detected at different time periods after obtaining the target classification result of the object to be detected; The device further includes: The second determining module is used to obtain the changing trend of the object to be detected based on the target classification results of the object to be detected in different time periods; and to determine the risk score of the object to be detected based on the changing trend.

[0145] In some embodiments of this application, the apparatus further includes: The processing module is configured to, after acquiring N skin spectral images, perform contrast enhancement processing on each skin spectral image to obtain an enhanced skin spectral image; perform filtering processing on the enhanced skin spectral image to obtain a filtered skin spectral image; and perform morphological restoration on the filtered skin spectral image to obtain a target skin spectral image. The detection module can specifically be used for: Based on N target skin spectral images, the object to be detected is detected to obtain the target classification result of the object to be detected.

[0146] The information detection device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0147] The information detection device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0148] The information detection device provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0149] Optionally, such as Figure 5 As shown, this application embodiment also provides an electronic device 500, including a processor 501 and a memory 502. The memory 502 stores a program or instructions that can run on the processor 501. When the program or instructions are executed by the processor 501, they implement the various steps of the above-described information detection method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0150] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0151] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0152] The electronic device 600 includes, but is not limited to, components such as: radio frequency unit 601, network module 602, audio output unit 603, input unit 604, sensor 605, display unit 606, user input unit 607, interface unit 608, memory 609, and processor 610.

[0153] Those skilled in the art will understand that the electronic device 600 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 610 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0154] The processor 610 is used to acquire N skin spectral images, each skin spectral image being acquired when light is emitted from a light source, and each of the N skin spectral images contains the object to be detected; based on the N skin spectral images, the object to be detected is detected to obtain the target classification result of the object to be detected.

[0155] Thus, the solution in this application embodiment can be applied to an electronic device integrating N light-emitting sources. Each light-emitting source emits light in one wavelength band. By detecting the object to be detected based on the skin spectral image acquired when each light-emitting source emits light, the target classification result of the object to be detected can be obtained. In this application embodiment, the acquired image is a multispectral image. Since each skin spectral image is a reflected light image of the object to be detected acquired in a different wavelength band, it helps to reveal the characteristics of the object to be detected in different tissue layers. Thus, when detecting the object to be detected based on the multispectral image, it can be detected from the characteristics of the object to be detected contained in different tissue layers, rather than relying solely on the appearance boundary of the object to be detected. Thus, the solution in this application embodiment improves the detection accuracy of the object to be detected.

[0156] Optionally, the processor 610 is further configured to register the N skin spectral images to obtain N registered skin spectral images; fuse the N registered skin spectral images to obtain a fused skin spectral image; segment the object to be detected in the fused skin spectral image to obtain a target detection object; and classify the target detection object to obtain a target classification result of the target object to be detected.

[0157] In this way, by registering N skin spectral images, N registered skin spectral images are obtained. Then, the N registered skin spectral images are fused to obtain a fused skin spectral image. The target object is extracted from the fused skin spectral image. Thus, the target object exists in each skin tissue layer, which helps to reveal the characteristics of the target object in different tissue layers. Therefore, when detecting the target object based on this multispectral image, it can be detected from the characteristics of the target object contained in different tissue layers, rather than relying solely on the appearance boundary of a single target object, thereby improving the detection accuracy of the target object.

[0158] Optionally, the N light sources include light sources emitting R-band light, G-band light, B-band light, and near-infrared (NIR) light; the N skin spectral images include R-band skin spectral images, G-band skin spectral images, B-band skin spectral images, and NIR skin spectral images; the processor 610 is further configured to determine a reference skin spectral image among the R-band, G-band, B-band, and NIR skin spectral images; match the pixels in the reference skin spectral images with the pixels in each skin spectral image to be matched, obtaining a transformation matrix between the reference skin spectral image and each skin spectral image to be matched, wherein the skin spectral images to be matched are skin spectral images other than the reference skin spectral image among the N skin spectral images; determine a target transformation matrix based on the transformation matrix between the reference skin spectral images and each skin spectral image to be matched; and based on the target transformation matrix, transform each skin spectral image to be matched into the coordinate system corresponding to the reference skin spectral image, obtaining the registered N skin spectral images.

[0159] Thus, by unifying N skin spectral images to the same reference image coordinate system, the slight angle changes or displacement errors caused by hand tremors during the shooting process of the N skin spectral images can be resolved, thereby avoiding the impact of errors caused by slight angle errors or hand tremors on the detection accuracy of the object to be detected. The solution of this application improves the detection accuracy of the object to be detected.

[0160] Optionally, the processor 610 is further configured to stitch the registered N skin spectral images along the channel dimension to obtain a stitched image, wherein the stitched image is an N-channel spectral image; and to perform weighted calculation on the image features of each channel in the stitched image according to the weight of each channel in the N channels to obtain a fused skin spectral image.

[0161] Since different spectral channels carry different biological tissue information and are affected by noise to varying degrees, some channels may have clear signals in certain areas while being noisy in others. Therefore, by stitching together the registered N skin spectral images along the channel dimension, a stitched image can be obtained. Then, based on the weight of each channel in the N channels, the image features of each channel in the stitched image are weighted to obtain a fused skin spectral image. In this way, by giving greater weight to channels with high signal-to-noise ratio and rich information, while suppressing channels with high noise, redundant information, or ineffective information, the final fused image can integrate the useful information of all channels, thus obtaining a fused skin spectral image that is clearer, less noisy, and richer in information than any single channel.

[0162] Optionally, the processor 610 is further configured to extract skin color regions from the fused skin spectral image to obtain a mask image corresponding to the fused skin spectral image; and based on the mask image, segment the object to be detected in the fused skin spectral image to obtain a target detection object.

[0163] Thus, by extracting the skin color region from the fused skin spectral image, a mask image corresponding to the fused skin spectral image is obtained. Based on the mask image, the target object in the fused skin spectral image can be segmented, which can improve the segmentation accuracy of the target object.

[0164] Optionally, the processor 610 is further configured to segment the object to be detected in the fused skin spectral image based on the mask image to obtain a candidate region of the object to be detected; and to filter the connected components in the candidate region of the object to obtain the target object to be detected.

[0165] In this way, by segmenting the target object in the fused skin spectral image based on the mask image, the candidate regions of the target object are obtained first. Then, the connected components in the candidate regions of the target object are filtered to obtain the target target object, instead of directly taking the target object in the candidate regions of the target object as the final target target object. This improves the accuracy of target object extraction and thus improves the accuracy of target object classification results.

[0166] Optionally, the processor 610 is further configured to, when the number of connected components is one, use the detection object in the minimum bounding rectangle of the connected component as the target detection object; and when the number of connected components is at least two, use the detection object in the minimum bounding rectangle of the target connected component as the target detection object, wherein the target connected component is one of the at least two connected components.

[0167] In this way, the main extraction region of the mole is determined based on the number of connected components in the candidate region of the detection object, and the target detection object is extracted based on the main extraction region, instead of directly extracting the target detection object from the entire candidate region of the detection object. This reduces the range of the target detection object extraction region, reduces the amount of computation, and improves the extraction accuracy of the target detection object.

[0168] Optionally, the processor 610 is further configured to, when the number of connected components is at least two, determine a score for each connected component based on the attribute information of each connected component, wherein the attribute information of a connected component includes at least the area, aspect ratio, roundness, and color density of the connected component; and determine the connected component with the highest score among the scores of each connected component as the target connected component.

[0169] Thus, when there are at least two connected components, the connected component with the highest score is selected based on the area, aspect ratio, roundness, and color density of each connected component, and this component is used as the target connected component. This target connected component contains the largest area and the largest color density of the target detection object, which can better extract the target detection object and improve the accuracy of the target detection object extraction.

[0170] Optionally, the processor 610 is further configured to input the mask image and the fused skin spectral image into an image segmentation model to segment the object to be detected in the fused skin spectral image to obtain the target detection object; the loss function of the image segmentation model adopts a binary cross loss function and a Dice loss function.

[0171] Thus, by segmenting the target object in the fused skin spectral image using an image segmentation model, rather than through manual annotation and extraction, the efficiency and accuracy of the target object segmentation are improved. Furthermore, the image segmentation model employs a composite loss function, enhancing its robustness.

[0172] Optionally, the processor 610 is further configured to acquire the target classification results of the object to be detected at different time periods; obtain the changing trend of the object to be detected based on the target classification results of the object to be detected at different time periods; and determine the risk score of the object to be detected based on the changing trend.

[0173] In this way, by analyzing the changing trends of the target classification results of the object to be detected in different time periods, the risk score of the object to be detected can be obtained. This allows for the analysis of the changing trends of the object to be detected in different time periods, enabling dynamic prediction of the development trajectory of the object to be detected, and assisting in judging the risk progress of the object to be detected, so that users can take corresponding measures in a timely manner.

[0174] Optionally, the processor 610 is further configured to: perform contrast enhancement processing on each skin spectral image to obtain an enhanced skin spectral image; perform filtering processing on the enhanced skin spectral image to obtain a filtered skin spectral image; perform morphological restoration on the filtered skin spectral image to obtain a target skin spectral image; and detect the object to be detected based on N target skin spectral images to obtain a target classification result for the object to be detected.

[0175] Thus, for each skin spectral image, preprocessing can be performed to eliminate the influence of interference factors such as uneven skin color, complex background, and hair occlusion, thereby improving the detection accuracy of the object to be detected.

[0176] It should be understood that, in this embodiment, the input unit 604 may include a graphics processing unit (GPU) 6041 and a microphone 6042. The GPU 6041 processes image data of still images or videos obtained by an image capture device (such as a color camera) in video capture mode or image capture mode. The display unit 606 may include a display panel 6061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 607 includes at least one of a touch panel 6071 and other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0177] The memory 609 can be used to store software programs and various data. The memory 609 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 609 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 609 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0178] Processor 610 may include one or more processing units; optionally, processor 610 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 610.

[0179] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described information detection method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0180] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0181] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described information detection method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0182] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0183] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the information detection method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0184] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0185] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0186] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An information detection method characterized by comprising: A method is executed by an electronic device, the electronic device is integrated with N light emitting sources, one light emitting source is used to emit light of one wave band, the method comprises: Obtaining N skin spectrum images, one skin spectrum image is collected when one light emitting source emits light, the N skin spectrum images all contain a to-be-detected object; According to the N skin spectrum images, the to-be-detected object is detected to obtain a target classification result of the to-be-detected object.

2. The method of claim 1, wherein, According to the N skin spectrum images, the to-be-detected object is detected to obtain a target classification result of the to-be-detected object, comprising: Registering the N skin spectrum images to obtain registered N skin spectrum images; Fusing the registered N skin spectrum images to obtain a fused skin spectrum image; Segmenting the to-be-detected object in the fused skin spectrum image to obtain a target detection object; Classifying the target detection object to obtain a target classification result of the target to-be-detected object.

3. The method of claim 2, wherein, The N light emitting sources include a light emitting source emitting R wave band light, a light emitting source emitting G wave band light, a light emitting source emitting B wave band light, and a light emitting source emitting near-infrared spectrum NIR light, and the N skin spectrum images include R light skin spectrum images, G light skin spectrum images, B light skin spectrum images, and NIR light skin spectrum images; The registration of the N skin spectrum images to obtain the registered N skin spectrum images comprises: Determining a reference skin spectrum image in the R light skin spectrum image, the G light skin spectrum image, the B light skin spectrum image, and the NIR light skin spectrum image; Matching the pixel points in the reference skin spectrum image with the pixel points in each to-be-matched skin spectrum image to obtain a transformation matrix between the reference skin spectrum image and each to-be-matched skin spectrum image, the to-be-matched skin spectrum image being a skin spectrum image other than the reference skin spectrum image in the N skin spectrum images; According to the transformation matrix between the reference skin spectrum image and each to-be-matched skin spectrum image, a target transformation matrix is determined; Based on the target transformation matrix, each to-be-matched skin spectrum image is converted into a coordinate system corresponding to the reference skin spectrum image to obtain the registered N skin spectrum images.

4. The method of claim 2, wherein, The fusion of the registered N skin spectrum images to obtain the fused skin spectrum image comprises: Splicing the registered N skin spectrum images in the channel dimension to obtain a spliced image, the spliced image being an N-channel spectrum image; According to the weight of each channel in the N channels, the image features of each channel in the spliced image are weighted and calculated to obtain the fused skin spectrum image.

5. The method of claim 2, wherein, The segmentation of the to-be-detected object in the fused skin spectrum image to obtain the target detection object comprises: Extracting a skin color region in the fused skin spectrum image to obtain a mask image corresponding to the fused skin spectrum image; Segment the to-be-detected object in the fused skin spectrum image based on the mask image to obtain a target detection object.

6. The method of claim 5, wherein, The method further includes: Segment the to-be-detected object in the fused skin spectrum image based on the mask image to obtain a target detection object. Segment the to-be-detected object in the fused skin spectrum image based on the mask image to obtain a target detection object.

7. The method of claim 6, wherein, Screen the connected domains in the detection object candidate region to obtain the target detection object. The method further includes: In a case where the number of connected domains is one, take the detection object in the minimum circumscribed rectangle of the connected domain as the target detection object.

8. The method of claim 7, wherein, In a case where the number of connected domains is at least two, take the detection object in the minimum circumscribed rectangle of a target connected domain as the target detection object, the target connected domain being one of the at least two connected domains. In a case where the number of connected domains is at least two, the method further includes: Determine a score of each connected domain according to attribute information of each connected domain, the attribute information of a connected domain at least including an area, an aspect ratio, a circularity, and a color density of the connected domain; 9. The method of claim 5, wherein, Determine a target connected domain as the connected domain with the highest score among the scores of the connected domains. The method further includes: Input the mask image and the fused skin spectrum image into an image segmentation model to segment the to-be-detected object in the fused skin spectrum image to obtain a target detection object.

10. The method of claim 1, wherein, The loss function of the image segmentation model adopts a binary cross-entropy loss function and a Dice loss function. After obtaining the target classification result of the to-be-detected object, the method further includes: Obtain the target classification result of the to-be-detected object in different time periods; Obtain a change trend of the to-be-detected object according to the target classification result of the to-be-detected object in different time periods; 11. The method of claim 1, wherein, Determine a risk score of the to-be-detected object according to the change trend. After obtaining the N skin spectrum images, the method further includes: For each skin spectrum image, perform contrast enhancement processing on the skin spectrum image to obtain an enhanced skin spectrum image; Perform filtering processing on the enhanced skin spectrum image to obtain a filtered skin spectrum image; Perform morphological repair on the filtered skin spectrum image to obtain a target skin spectrum image. The method further includes:

12. An information detection device characterized by comprising: Detect the to-be-detected object according to the N target skin spectrum images to obtain a target classification result of the to-be-detected object. The device is applied to an electronic device, the electronic device being integrated with N light-emitting light sources, one light-emitting light source being used to emit light rays of one wave band, and the device including: An acquisition module is configured to acquire N skin spectrum images, one skin spectrum image is collected when a light-emitting light source emits light, and the N skin spectrum images all contain a to-be-detected object; A detection module is configured to detect the to-be-detected object according to the N skin spectrum images, and obtain a target classification result of the to-be-detected object.

Images