Spot classification method and device, computer device and storage medium

Abstract

The application relates to a spot classification method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring point cloud data sets and an original image collected for a target part of a target object; determining the position of a spot on the target part and the contour of the spot in the original image; generating a color mode image according to the original image; determining the position of the spot on the target part and the contour of the spot in the color mode image according to the position of the spot on the target part and the contour of the spot in the original image; determining the color information of the spot on the target part in the color mode image according to the position of the spot on the target part and the contour of the spot in the color mode image; determining the category of the spot on the target part in the original image according to the color information of the spot on the target part in the color mode image; and controlling a laser emitter to perform laser removal on the laser-removable spot on the target part. The method can accurately remove skin spots.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer device, and storage medium for classifying spots. Background Technology

[0002] With the continuous development of technology and the continuous improvement of people's living standards, the means to improve and enhance skin through technological means are becoming increasingly diverse. Among them, the improvement and enhancement of blemishes, which have the greatest impact on skin condition, has also received widespread attention, and methods for detecting and analyzing skin blemishes have been further optimized.

[0003] However, a major problem with current methods for detecting and analyzing skin spots is that the identification of skin spot types remains relatively vague. The detection results are not precise enough, making it impossible to determine which specific and effective removal method should be adopted, and thus failing to provide a reasonable basis for subsequent removal of skin spots. Summary of the Invention

[0004] Therefore, it is necessary to provide a spot classification method, apparatus, computer device, and computer-readable storage medium that can improve the accuracy of skin spot classification, addressing the aforementioned technical problems.

[0005] Firstly, this application provides a spot classification method, including:

[0006] Acquire point cloud datasets and original images of the target parts of the target object;

[0007] Based on the point cloud dataset and the original image, determine the position and outline of the spots on the target region in the original image;

[0008] Generate a color mode image based on the original image;

[0009] Based on the position and outline of the spots on the target area in the original image, determine the position and outline of the spots on the target area in the color mode image;

[0010] Based on the position and outline of the spots on the target area in the color mode image, determine the color information of the spots on the target area in the color mode image;

[0011] Based on the color information of the spots on the target area in the color mode image, determine the category of the spots on the target area in the original image;

[0012] Based on the category of spots on the target area in the original image, laser-removable spots are identified from the spots on the target area in the original image, calibration parameters are obtained, and a laser emitter is controlled to remove the laser-removable spots on the target area according to the calibration parameters. The calibration parameters are the conversion coefficient between the position of the laser emitter and the position of the spots on the target area.

[0013] Secondly, this application also provides a spot classification device, comprising:

[0014] The acquisition module is used to acquire point cloud datasets and original images of the target parts of the target object;

[0015] The first determining module is used to determine the position and outline of the spots on the target region in the original image based on the point cloud dataset and the original image;

[0016] The generation module is used to generate a color mode image based on the original image;

[0017] The second determining module is used to determine the position and outline of the spots on the target area in the color mode image based on the position and outline of the spots on the target area in the original image.

[0018] The third determining module is used to determine the color information of the spots on the target part in the color mode image based on the position and outline of the spots on the target part in the color mode image;

[0019] The fourth determining module is used to determine the category of the spots on the target area in the original image based on the color information of the spots on the target area in the color mode image;

[0020] The laser removal module is used to determine laser-removable spots among the spots on the target area in the original image according to the category of spots on the target area, obtain calibration parameters, and control the laser emitter to remove the laser-removable spots on the target area according to the calibration parameters. The calibration parameters are the conversion coefficient between the position of the laser emitter and the position of the spots on the target area.

[0021] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0022] Acquire point cloud datasets and original images of the target parts of the target object;

[0023] Based on the point cloud dataset and the original image, determine the position and outline of the spots on the target region in the original image;

[0024] Generate a color mode image based on the original image;

[0025] Based on the position and outline of the spots on the target area in the original image, determine the position and outline of the spots on the target area in the color mode image;

[0026] Based on the position and outline of the spots on the target area in the color mode image, determine the color information of the spots on the target area in the color mode image;

[0027] Based on the color information of the spots on the target area in the color mode image, determine the category of the spots on the target area in the original image;

[0028] Based on the category of spots on the target area in the original image, laser-removable spots are identified from the spots on the target area in the original image, calibration parameters are obtained, and a laser emitter is controlled to remove the laser-removable spots on the target area according to the calibration parameters. The calibration parameters are the conversion coefficient between the position of the laser emitter and the position of the spots on the target area.

[0029] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0030] Acquire point cloud datasets and original images of the target parts of the target object;

[0031] Based on the point cloud dataset and the original image, determine the position and outline of the spots on the target region in the original image;

[0032] Generate a color mode image based on the original image;

[0033] Based on the position and outline of the spots on the target area in the original image, determine the position and outline of the spots on the target area in the color mode image;

[0034] Based on the position and outline of the spots on the target area in the color mode image, determine the color information of the spots on the target area in the color mode image;

[0035] Based on the color information of the spots on the target area in the color mode image, determine the category of the spots on the target area in the original image;

[0036] Based on the category of spots on the target area in the original image, laser-removable spots are identified from the spots on the target area in the original image, calibration parameters are obtained, and a laser emitter is controlled to remove the laser-removable spots on the target area according to the calibration parameters. The calibration parameters are the conversion coefficient between the position of the laser emitter and the position of the spots on the target area.

[0037] The aforementioned spot classification method, apparatus, computer equipment, and storage medium acquire a point cloud dataset and an original image of the target area of ​​the target object; determine the position and outline of the spots on the target area in the original image based on the point cloud dataset and the original image; generate a color mode image based on the original image; determine the position and outline of the spots on the target area in the color mode image based on the position and outline of the spots on the target area in the original image; determine the color information of the spots on the target area in the color mode image based on the position and outline of the spots on the target area in the color mode image; determine the category of the spots on the target area in the original image based on the color information of the spots on the target area in the color mode image; identify laser-removable spots among the spots on the target area in the original image based on the category of the spots on the target area in the original image; obtain calibration parameters; and control a laser emitter to perform laser removal on the laser-removable spots on the target area based on the calibration parameters. The calibration parameters are the conversion coefficients between the position of the laser emitter and the position of the spots on the target area. This makes it clearer and more accurate to determine the type of skin spots, and the test results are more precise and detailed, providing a reasonable basis for choosing the appropriate removal method for the skin spots in the subsequent process. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a diagram illustrating the application environment of the spot classification method in one embodiment;

[0040] Figure 2 This is a flowchart illustrating a spot classification method in one embodiment;

[0041] Figure 3 This is a flowchart illustrating the spot classification method in another embodiment;

[0042] Figure 4 This is a structural block diagram of a spot classification device in one embodiment;

[0043] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0045] The spot classification method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located in the cloud or on other network servers. Terminal 102 generates a blob classification request and then sends it to server 104 so that server 104 can determine the category of blobs on the target area in the original image. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0046] In one exemplary embodiment, such as Figure 2 As shown, a spot classification method is provided, which can be applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 202 to 212. Wherein:

[0047] Step 202: Obtain the point cloud dataset and original image of the target part of the target object.

[0048] In this context, the target object refers to the object being classified as a blob, typically the human body. The target body part refers to a part of the human body within the target object (the human body being classified as a blob), typically the face. A point cloud dataset is a collection of point cloud data, consisting of a large number of discrete three-dimensional points. Each point contains coordinate information in three-dimensional space and possible other attributes, such as color and normals. Point cloud data is typically obtained through laser scanning, 3D cameras, or other sensors and is used to describe the shape and structure of an object's surface. A raw image refers to an image of an object captured by a camera or sensor without any processing or modification. These images can be acquired through cameras, scanners, or other types of sensors.

[0049] Specifically, in the process of classifying spots on a human face, the first step is to collect point cloud data of the face. The collection of point cloud data is called a point cloud dataset. Then, the original image of the face is acquired. It is easy to understand that the specific methods for collecting the point cloud data and the original image are not limited and can be set according to actual needs. Optional methods include using 3D scanners, structured light projection, laser scanning, and stereo cameras to collect the point cloud data. Alternatively, the original image of the face can be acquired using data cameras, infrared cameras, optical scanners, etc.

[0050] Step 204: Based on the point cloud dataset and the original image, determine the location and outline of the spots on the target area in the original image.

[0051] In this application, "spot" refers to spots on the human face, including at least freckles, melasma, strawberry spots, and age spots. The location of a spot refers to its specific position on the face, such as the chin. The outline of a spot refers to the lines or shape of its outer boundary or shape, used to describe its shape and structure. The shape and size of each spot can be delineated using lines or edges. The outline of a spot defines its overall appearance and helps in understanding its morphological characteristics and size proportions. In this application, the outline of a spot typically refers to its outer contour.

[0052] Specifically, and easily understood, the specific method for determining the location and contour of spots on a human face in the original image based on the point cloud dataset and the original image is not limited and can be set according to actual needs. Optionally, the contour of spots on the human face in the original image can be obtained through edge detection to obtain the outer contours of multiple detected spots. After obtaining the outer contour of each spot on the human face, the specific location of each spot on the human face can be determined based on its outer contour. The specific method for obtaining the specific location of each spot on the human face is not limited and can be set according to actual needs. Optionally, the specific location of each spot on the human face can be determined using the minimum bounding rectangle method or the minimum bounding circle method.

[0053] Step 206: Generate a color mode image based on the original image.

[0054] Color mode is one of the methods used in image processing to describe and represent color; it defines how digital values ​​are mapped to actual colors. Different color modes use different color representation methods. This application does not limit the specific form of the color mode image and can set it according to actual needs. Optionally, the specific form of the color mode image can be RGB (red, green, blue), CMYK (cyan, magenta, yellow, black), HSV (hue, saturation, brightness), or Lab (a color space based on human visual perception, including brightness L and two chromaticity components a and b).

[0055] Specifically, for the human face used for spot classification, a color pattern image of the face is acquired. It is easy to understand that the specific method for acquiring the color pattern image of the face is not limited and can be set according to actual needs. Optionally, the color pattern image of the face can be acquired using image processing software or a programming voice library.

[0056] Step 208: Determine the position and outline of the spots on the target area in the color mode image based on the position and outline of the spots on the target area in the original image.

[0057] Specifically, considering the need to analyze facial spots in a color mode image, it is necessary to first determine the specific location and outline of each spot in the color mode image. The specific location and outline of each spot in the original image have already been determined in the steps described above. Therefore, it is only necessary to transform and map the known specific location and outline of each spot in the original image to the specific location and outline of each spot in the color mode image.

[0058] Step 210: Determine the color information of the spots on the target area in the color mode image based on the position and outline of the spots.

[0059] This application does not limit the specific type of color information and can set it according to actual needs. Optionally, for RGB (red, green, blue) color mode images, the color information of each spot refers to the RGB value, specifically including the red channel value, green channel value, and blue channel value corresponding to each spot. The method for determining the color information corresponding to each spot is also not specifically limited and can be set according to actual needs. Optionally, RGB values ​​can be obtained using image processing software tools, online color pickers, or by reading image files using a programming language.

[0060] Specifically, after determining the position and outline of each spot in the color mode image, an online color sampling tool is used to perform color analysis on each spot in the color mode image, and finally the RGB values ​​of each spot in the color mode image, namely the red channel value, green channel value and blue channel value, are obtained.

[0061] Step 212: Determine the category of the spots on the target area in the original image based on the color information of the spots on the target area in the color mode image.

[0062] This application does not limit the specific types of spots; they can be set according to actual needs. Optionally, spot categories include at least freckles, melasma, sunspots, and age spots. Since the color of the spot is the most crucial factor in determining its category, the category of each spot can be determined based on its color information.

[0063] Specifically, taking an RGB image as an example, after obtaining the red, green, and blue channel values ​​for each spot on a human face, these values ​​are converted into corresponding color attributes. Color attributes refer to the characteristics that describe and classify colors; these attributes help in understanding and distinguishing different colors. Then, based on the color attributes of each spot on the human face, the spot category for each spot is analyzed.

[0064] Step 214: Based on the category of spots on the target area in the original image, identify the spots that can be removed by laser, obtain calibration parameters, and control the laser emitter to remove the spots that can be removed by laser based on the calibration parameters. The calibration parameters are the conversion coefficient between the position of the laser emitter and the position of the spots on the target area.

[0065] Specifically, based on the color characteristics of the spots in the color pattern image, image recognition algorithms or databases of known spot types are used to determine the category of the spots on the target area in the original image. For safe spot categories that can be treated with laser, the spots to be removed are marked in the original image. The calibration parameters required for effective treatment of the target spots by the laser emitter are obtained, including the spatial position and angle adjustment of the laser emitter, as well as the conversion coefficient between the position of the target spot and the laser emitter, to ensure that the laser can be accurately projected onto the spot area.

[0066] In one example, the laser treatment of skin spots mainly includes the following steps: First, spot information identification and localization. A vision module (such as a high-resolution camera) is used to acquire images of the skin spots, and image processing algorithms are used to identify the location, shape, and extent of the spots. Next, system calibration is performed to determine the transformation relationship from the visual space to the laser coordinate system. This calibration process may include physical calibration experiments or software algorithm calculations to obtain the calibration parameters needed to map the spot location data captured by the vision module to the actual working space of the laser emitter.

[0067] Next comes information matching and transformation. Using the aforementioned calibration parameters, the identified spot position information is transformed to ensure it accurately corresponds to the laser's working coordinate system. This transformation process ensures that the laser can achieve precise positioning based on the spot position data provided by the vision module.

[0068] Next is the operation of the laser treatment module. This module is equipped with a light source capable of generating high-energy, high-frequency laser beams. These beams have sufficient energy density to target pigmented spots on the skin surface. Based on the preset treatment plan and the converted spot location information, the laser module can emit the laser at the appropriate time and angle, ensuring the laser beam directly hits and acts on the target spot area.

[0069] Finally, laser treatment is performed. Under precise laser irradiation, the pigmented tissue within the spot absorbs laser energy of a specific wavelength and is destroyed or decomposed, thereby achieving the therapeutic goal.

[0070] Because safety and effectiveness must be considered during treatment, parameters such as laser energy output, pulse width, and frequency are individually adjusted based on the type of blemishes and individual skin characteristics. Furthermore, by comprehensively utilizing image recognition, coordinate transformation, and laser treatment principles, precise localization of skin blemishes and efficient, safe laser treatment are achieved. Through precise information transmission and laser application, laser therapy becomes more refined and individualized, significantly improving the effectiveness of blemish removal treatment and enhancing the patient experience.

[0071] In one embodiment, blob detection is performed on the original image to obtain blob contours; the minimum circular bounding box of the blob contours is determined; the point cloud dataset is mapped onto the original image, and the point cloud data mapped to the minimum circular bounding box is extracted; the position of the blob on the target area in the original image is determined based on the point cloud data; and the contour of the blob on the target area in the original image is determined based on the blob contours.

[0072] Specifically, the process begins with blob detection on the original image. An edge detection algorithm is used to locate multiple blobs in the image and obtain their contour information. Then, based on these blob contours, the smallest circular bounding box (BBO) containing each blob contour is determined; that is, each blob contour is enclosed by the smallest possible circle. Next, the point cloud dataset is mapped onto the original image, that is, the point cloud data with 3D coordinate information is mapped onto a 2D image. By extracting the point cloud data mapped to each BBO, the positions of the corresponding blobs in the original image are determined. Based on the extracted point cloud data, the positions of the blobs on the target area in the original image are determined, thus locating the specific positions of the blobs in the image. Finally, based on the previously obtained blob contours, the shape contours of the blobs on the target area are determined, which helps to describe the shape characteristics of the blobs.

[0073] By combining image processing and point cloud data processing, information from different data sources can be fully utilized, which helps improve the accuracy of spot location and shape. Furthermore, using the smallest circular bounding box can more accurately describe the shape of the spots, avoiding the influence of the complexity and variations in spot shape.

[0074] In one embodiment, the original image is divided into multiple grids, and the image features of each grid are extracted; based on the image features of each grid, a feature vector for each grid is generated, and based on the feature vector of each grid, target grids containing blobs are selected from the multiple grids; the original image is segmented according to the positional relationship between the target grids to obtain an original segmented image; the original segmented image is binarized to obtain a binary image; based on the pixel values ​​of each pixel in the binary image, initial contour detection is performed on the binary image to obtain an initial contour point set corresponding to the binary image; based on the initial contour point set corresponding to the binary image, the blob contour is determined.

[0075] Specifically, the original image is first divided into multiple small grids, and image features are extracted from each grid. Based on the extracted image features, corresponding feature vectors are generated, and these feature vectors are used to filter out target grids containing blobs. Then, the original image is segmented using the positional relationships between the target grids, resulting in at least one original segmented image. Each original segmented image is then binarized, converting it into a black-and-white binary image. Next, for each binary image, initial contour detection is performed based on the pixel values ​​of each pixel, obtaining an initial contour point set. Finally, based on the obtained initial contour point set, the contour of each blob in the image is determined.

[0076] Because of the grid partitioning and feature extraction, it can adapt to images of different sizes and complexities, and the feature extraction method can be flexibly adjusted to suit different types of blobs. Furthermore, binarization and contour detection can make the blob contours clearer, which is helpful for further blob analysis and processing.

[0077] In one embodiment, for every two initial contour points in the initial contour point set corresponding to the binary image, a line segment is determined with the two initial contour points as endpoints, and if there are other initial contour points in the line segment, the other initial contour points are deleted; based on the remaining initial contour points in the initial contour point set corresponding to the binary image, the spot contour is determined.

[0078] Specifically, first, every two points are selected from the initial contour point set, and the line segment formed by these two points is determined. Then, other initial contour points are searched within the line segment; if any are found, they are deleted. This method connects adjacent points in the initial contour point set and removes points that do not conform to the contour characteristics. After removing other initial contour points, the remaining points form the blob contour. Note that this blob contour may no longer exist as a line segment, but rather as a series of adjacent points.

[0079] By connecting the points in the initial contour point set and removing points that do not conform to the contour characteristics, the blob contours in the binary image can be effectively extracted, which is beneficial for subsequent blob analysis and processing. Furthermore, by constructing line segments and deleting redundant points, the blob contours can be made clearer and simpler, facilitating subsequent processing and analysis. This method is relatively flexible, applicable to blob contour extraction of different shapes and sizes, and suitable for different types of binary images.

[0080] In one embodiment, the point cloud coordinate system where the point cloud dataset is located is obtained; the original image coordinate system where the original image is located is obtained; the transformation matrix between the point cloud coordinate system and the original image coordinate system is determined; and the point cloud dataset in the point cloud coordinate system is mapped to the original image in the original image coordinate system according to the transformation matrix.

[0081] Specifically, the first step is to clearly define the coordinate system of the point cloud dataset and the coordinate system of the original image. Understandably, the point cloud coordinate system is typically a three-dimensional coordinate system, while the original image coordinate system is a two-dimensional pixel coordinate system. Then, based on the relative positions of the point cloud and original image coordinate systems, their transformation relationship can be determined, usually represented by a first transformation matrix. This transformation matrix may involve operations such as translation, rotation, and scaling to ensure that the point cloud dataset is correctly mapped to the original image coordinate system. Finally, based on the determined first transformation matrix, the coordinates of each point in the point cloud dataset can be transformed accordingly, mapping it to its position in the original image coordinate system. Ultimately, this allows for the visualization of the point cloud dataset within the two-dimensional original image or for corresponding analysis with the original image.

[0082] Mapping point cloud data onto the original image allows for the visualization of the point cloud data in a two-dimensional image, facilitating intuitive observation and analysis of the spatial distribution of the point cloud data. Furthermore, the mapped point cloud data corresponds to the original image, making it easier to perform correlation analysis between the point cloud data and the image data.

[0083] In one embodiment, the center position of the smallest circular bounding box is determined; target point cloud data is filtered from the point cloud data mapped to the smallest circular bounding box based on the position information of the point cloud data mapped to the smallest circular bounding box; and the position of the spot on the target part in the original image is determined based on the position information of the target point cloud data.

[0084] Specifically, firstly, based on the mapped point cloud data and the information of the smallest circular bounding boxes, the point cloud data mapped to each smallest circular bounding box is extracted. For each smallest circular bounding box, its center position is determined, which can serve as a reference point for localization. Then, based on the positional information of the point cloud data mapped to each smallest circular bounding box, the required target point cloud data is filtered out. Algorithms or rules may be used to determine which point cloud data belong to spots on the target area. Finally, based on the positional information of the filtered target point cloud data, their positions in the original image are determined.

[0085] By utilizing the spatial information of point cloud data and combining it with the positional information of the minimum circular bounding box, the location of spots in the original image can be determined more accurately, thus improving the localization accuracy. Furthermore, the use of the minimum circular bounding box allows for adaptation to target areas of various shapes and sizes, increasing the method's versatility and applicability.

[0086] In one embodiment, the correspondence between a first pixel in the original image and a second pixel in the color mode image is determined; based on the correspondence between the first pixel in the original image and the second pixel in the color mode image, and the position and outline of the spots on the target area in the original image, the position and outline of the spots on the target area are determined.

[0087] Specifically, the correspondence between pixels in the original image and the color mode image is first established. Through operations such as coordinate transformation and color space conversion, the correspondence between pixels at the same location in the two images is ensured. Then, based on the established pixel correspondence, the position and contour information of the corresponding spots in the color mode image are derived from the position and contour information of the spots in the original image. Through pixel mapping and interpolation, the position and contour information of the spots are reconstructed in the color mode image.

[0088] By establishing the pixel correspondence between the original image and the color mode image, the conversion of the spot position and outline can be guaranteed to be accurate, avoiding information misalignment or mismatch.

[0089] In one embodiment, the region location of the spots on the target area in the color mode image is determined based on the center position and outline of the spots; the red channel value, green channel value, and blue channel value of each pixel within the region location are determined; the average red channel value of each pixel within the region location is obtained by averaging the red channel value; the average green channel value of each pixel within the region location is obtained by averaging the green channel value; the average blue channel value of each pixel within the region location is obtained by averaging the blue channel value; and the color information of the spots on the target area in the color mode image is determined based on the average red channel value, average green channel value, and average blue channel value.

[0090] Specifically, firstly, the location of the spot is determined based on its center position and outline in the color mode image. Then, within the determined location, the red, green, and blue channel values ​​of each pixel are averaged to obtain the average red, green, and blue channel values ​​for that location. Finally, the color information of the spot is determined based on the average red, green, and blue channel values. For example, for a spot in the color mode image located at the tip of the nose on a human face, and whose outline has a radius of 0.5 mm, the location of the spot can be determined as a circular area with a radius of 0.5 mm located at the tip of the nose.

[0091] By averaging the color channel values ​​of each pixel within a given region, the overall color information of the spots can be effectively extracted, resulting in high accuracy and reliability. Furthermore, by analyzing the position and outline of the spots in the color mode image, the regional location of the spots can be precisely determined, enabling accurate color analysis of the spots.

[0092] In one embodiment, the brightness, saturation, and hue of the spots on the target area in the original image are determined based on the average red channel value, average green channel value, and average blue channel value of the spots on the target area in the color mode image; the brightness, saturation, and hue of the spots on the target area in the original image are then input into a spot classification model to obtain the category of the spots on the target area in the original image.

[0093] Specifically, the color features of each spot are first obtained by calculating the average red, green, and blue channel values. Then, brightness, saturation, and hue are calculated based on these features, reflecting the lightness, darkness, purity, and position of the color on the color wheel. Next, a spot classification model is built, using the brightness, saturation, and hue of each spot as input features. The model is trained using machine learning algorithms such as support vector machines, decision trees, and neural networks. Finally, the trained classification model is applied to the input spot data to classify each spot and obtain its corresponding spot category. These categories describe the features and attributes of the spots.

[0094] Since spot classification is based on color information, it avoids actual physical intervention in the target area, thus being non-invasive and suitable for fields requiring non-invasive analysis, such as medical imaging and biological imaging. Furthermore, by analyzing the color information of each spot, different types of spots can be distinguished more precisely, thereby improving the accuracy and precision of classification.

[0095] In one exemplary embodiment, such as Figure 3 As shown, Figure 3 This is a flowchart illustrating a spot classification method in another embodiment, including steps 302 to 312. Wherein:

[0096] Step 302: Obtain the point cloud dataset and original image of the target part of the target object;

[0097] Step 304: Perform blob detection on the original image to obtain blob contours; determine the minimum circular bounding box of the blob contours; map the point cloud dataset to the original image and extract the point cloud data mapped to the minimum circular bounding box; determine the position of the blob on the target area in the original image based on the point cloud data; determine the contour of the blob on the target area in the original image based on the blob contours.

[0098] Step 306: Generate a color mode image based on the original image;

[0099] Step 308: Determine the correspondence between the first pixel in the original image and the second pixel in the color mode image; based on the correspondence between the first pixel in the original image and the second pixel in the color mode image, and the position and outline of the spots on the target area in the original image, determine the position and outline of the spots on the target area in the color mode image.

[0100] Step 310: Based on the center position and outline of the spots on the target area in the color mode image, determine the region position of the spots on the target area in the color mode image; determine the red channel value, green channel value, and blue channel value of each pixel within the region position; average the red channel value corresponding to each pixel within the region position to obtain the average red channel value within the region position; average the green channel value corresponding to each pixel within the region position to obtain the average green channel value within the region position; average the blue channel value corresponding to each pixel within the region position to obtain the average blue channel value within the region position; determine the color information of the spots on the target area in the color mode image based on the average red channel value, average green channel value, and average blue channel value.

[0101] Step 312: Based on the average red channel value, average green channel value, and average blue channel value of the spots on the target area in the color mode image, determine the brightness, saturation, and hue of the spots on the target area in the original image; input the brightness, saturation, and hue of the spots on the target area in the original image into the spot classification model to obtain the category of the spots on the target area in the original image.

[0102] Step 314: Based on the category of spots on the target area in the original image, identify the spots that can be removed by laser, obtain calibration parameters, and control the laser emitter to remove the spots that can be removed by laser based on the calibration parameters. The calibration parameters are the conversion coefficient between the position of the laser emitter and the position of the spots on the target area.

[0103] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0104] Based on the same inventive concept, this application also provides a spot classification device for implementing the spot classification method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more of the spot classification device embodiments provided below can be found in the limitations of the spot classification method described above, and will not be repeated here.

[0105] In one exemplary embodiment, such as Figure 4 As shown, a spot classification device 400 is provided, including: an acquisition module 402, a first determination module 404, a generation module 406, a second determination module 408, a third determination module 410, a fourth determination module 412, and a laser removal module 414, wherein:

[0106] The acquisition module 402 is used to acquire point cloud datasets and original images collected from the target parts of the target object;

[0107] The first determining module 404 is used to determine the position and outline of the spots on the target part in the original image based on the point cloud dataset and the original image.

[0108] Generation module 406 is used to generate a color mode image based on the original image;

[0109] The second determining module 408 is used to determine the position and outline of the spots on the target area in the color mode image based on the position and outline of the spots on the target area in the original image.

[0110] The third determining module 410 is used to determine the color information of the spots on the target part of the color mode image based on the position and outline of the spots on the target part of the color mode image.

[0111] The fourth determining module 412 is used to determine the category of the spots on the target area in the original image based on the color information of the spots on the target area in the color mode image;

[0112] The laser removal module 414 is used to determine the laser-removable spots among the spots on the target area in the original image according to the category of spots on the target area, obtain calibration parameters, and control the laser emitter to remove the laser-removable spots on the target area according to the calibration parameters. The calibration parameters are the conversion coefficient between the position of the laser emitter and the position of the spots on the target area.

[0113] In one embodiment, the first determining module 404 is used to perform blob detection on the original image to obtain blob contours; determine the minimum circular bounding box of the blob contours; map the point cloud dataset to the original image and extract the point cloud data mapped to the minimum circular bounding box; determine the position of the blob on the target part in the original image based on the point cloud data; and determine the contour of the blob on the target part in the original image based on the blob contours.

[0114] In one embodiment, the first determining module 404 is used to divide the original image into multiple grids and extract the image features of each grid; generate a feature vector for each grid based on its image features, and select target grids containing blobs from the multiple grids based on the feature vectors of each grid; segment the original image according to the positional relationship between the target grids to obtain an original segmented image; perform binarization processing on the original segmented image to obtain a binary image; perform initial contour detection on the binary image based on the pixel values ​​of each pixel in the binary image to obtain an initial contour point set corresponding to the binary image; and determine the blob contour based on the initial contour point set corresponding to the binary image.

[0115] In one embodiment, the first determining module 404 is used to determine a line segment formed by the two initial contour points as endpoints for every two initial contour points in the initial contour point set corresponding to the binary image, and delete other initial contour points if there are other initial contour points in the line segment; and determine the spot contour based on the remaining initial contour points in the initial contour point set corresponding to the binary image.

[0116] In one embodiment, the first determining module 404 is used to obtain the point cloud coordinate system where the point cloud dataset is located; obtain the original image coordinate system where the original image is located; determine the transformation matrix between the point cloud coordinate system and the original image coordinate system; and map the point cloud dataset in the point cloud coordinate system to the original image in the original image coordinate system according to the transformation matrix.

[0117] In one embodiment, the first determining module 404 is used to determine the center position of the minimum circular bounding box; filter out target point cloud data from the point cloud data mapped to the minimum circular bounding box based on the position information of the point cloud data mapped to the minimum circular bounding box; and determine the position of the spots on the target part in the original image based on the position information of the target point cloud data.

[0118] In one embodiment, the second determining module 408 is used to determine the correspondence between a first pixel in the original image and a second pixel in the color mode image; and to determine the position and outline of the spots in the color mode image based on the correspondence between the first pixel in the original image and the second pixel in the color mode image, as well as the position and outline of the spots on the target area in the original image.

[0119] In one embodiment, the third determining module 410 is configured to: determine the region position of the spots on the target area in the color mode image based on the center position and outline of the spots; determine the red channel value, green channel value, and blue channel value of each pixel within the region position; average the red channel value corresponding to each pixel within the region position to obtain the average red channel value within the region position; average the green channel value corresponding to each pixel within the region position to obtain the average green channel value within the region position; average the blue channel value corresponding to each pixel within the region position to obtain the average blue channel value within the region position; and determine the color information of the spots on the target area in the color mode image based on the average red channel value, average green channel value, and average blue channel value.

[0120] In one embodiment, the fourth determining module 412 is used to determine the brightness, saturation, and hue of the spots on the target area in the original image based on the average red channel value, average green channel value, and average blue channel value of the spots on the target area in the color mode image; and input the brightness, saturation, and hue of the spots on the target area in the original image into the spot classification model to obtain the category of the spots on the target area in the original image.

[0121] Each module in the above-mentioned spot classification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0122] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data related to blob classification. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a blob classification method.

[0123] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0124] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0125] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0126] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0127] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0128] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0129] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for classifying spots, characterized in that, The method includes: Acquire point cloud datasets and original images of the target parts of the target object; Based on the point cloud dataset and the original image, determine the position and outline of the spots on the target region in the original image; Generate a color mode image based on the original image; Based on the position and outline of the spots on the target area in the original image, determine the position and outline of the spots on the target area in the color mode image; Based on the position and outline of the spots on the target area in the color mode image, determine the color information of the spots on the target area in the color mode image; Based on the color information of the spots on the target area in the color mode image, determine the category of the spots on the target area in the original image; Based on the category of spots on the target area in the original image, laser-removable spots are identified from the spots on the target area in the original image, calibration parameters are obtained, and a laser emitter is controlled to remove the laser-removable spots on the target area according to the calibration parameters. The calibration parameters are the conversion coefficient between the position of the laser emitter and the position of the spots on the target area.

2. The method according to claim 1, characterized in that, The step of determining the position and outline of the spots on the target region in the original image based on the point cloud dataset and the original image includes: Perform blob detection on the original image to obtain blob contours; The smallest circular bounding box that defines the outline of the spot; Map the point cloud dataset to the original image, and extract the point cloud data mapped to the smallest circular bounding box; Based on the point cloud data, determine the location of the spots on the target area in the original image; Based on the spotted outline, the outline of the spotted on the target region in the original image is determined.

3. The method according to claim 2, characterized in that, The step of performing blob detection on the original image to obtain blob contours includes: The original image is divided into multiple grids, and the image features of each grid are extracted. Based on the image features of each grid, a feature vector is generated for each grid, and based on the feature vector of each grid, the target grid containing spots is selected from the multiple grids; The original image is segmented according to the positional relationship between the target grids to obtain an original segmented image; The original segmented images are binarized to obtain binary images; Based on the pixel values ​​of each pixel in the binary image, initial contour detection is performed on the binary image to obtain the initial contour point set corresponding to the binary image; The blob contour is determined based on the initial contour point set corresponding to the binary image.

4. The method according to claim 3, characterized in that, Determining the blob contour based on the initial contour point set corresponding to the binary image includes: For every two initial contour points in the initial contour point set corresponding to the binary image, a line segment is determined with the two initial contour points as endpoints, and if there are other initial contour points in the line segment, the other initial contour points are deleted. The spot contour is determined based on the remaining initial contour points in the initial contour point set corresponding to the binary image.

5. The method according to claim 2, characterized in that, The step of mapping the point cloud dataset to the original image includes: Obtain the point cloud coordinate system in which the point cloud dataset is located; Obtain the original image coordinate system in which the original image is located; Determine the transformation matrix between the point cloud coordinate system and the original image coordinate system; The point cloud dataset in the point cloud coordinate system is mapped to the original image in the original image coordinate system according to the transformation matrix.

6. The method according to claim 2, characterized in that, Determining the location of the spots on the target region in the original image based on the point cloud data includes: Determine the center position of the minimum circular bounding box; Based on the position information of the point cloud data mapped to the minimum circular bounding box, the target point cloud data is filtered out from the point cloud data mapped to the minimum circular bounding box; Based on the location information of the target point cloud data, the location of the spots on the target region in the original image is determined.

7. The method according to claim 1, characterized in that, The step of determining the position and outline of the spots in the color mode image based on the position and outline of the spots on the target area in the original image includes: Determine the correspondence between the first pixel in the original image and the second pixel in the color mode image; Based on the correspondence between the first pixel in the original image and the second pixel in the color mode image, and the position and outline of the spots on the target area in the original image, the position and outline of the spots on the target area in the color mode image are determined.

8. The method according to claim 1, characterized in that, The position of the spot on the target area in the color mode image is the center position of the spot on the target area in the color mode image; determining the color information of the spot on the target area in the color mode image based on the position and outline of the spot includes: Based on the center position and outline of the spots on the target area in the color mode image, determine the region position of the spots on the target area in the color mode image; Determine the red channel value, green channel value, and blue channel value of each pixel within the specified region; The average red channel value of each pixel within the region is obtained by averaging the values ​​of the red channel within the region. The average green channel value of each pixel within the region is obtained by averaging the values ​​of the green channels within the region. The average blue channel value of each pixel within the region is obtained by averaging the values ​​of the blue channel within the region. The color information of the spots on the target area in the color mode image is determined based on the average red channel value, the average green channel value, and the average blue channel value.

9. The method according to claim 1, characterized in that, The step of determining the category of the spots on the target area in the original image based on the color information of the spots on the target area in the color mode image includes: Based on the average red channel value, average green channel value, and average blue channel value of the spots on the target area in the color mode image, determine the brightness, saturation, and hue of the spots on the target area in the original image; The brightness, saturation, and hue of the spots on the target area in the original image are input into the spot classification model to obtain the category of the spots on the target area in the original image.

10. A spot classification device, characterized in that, The device includes: The acquisition module is used to acquire point cloud datasets and original images of the target parts of the target object; The first determining module is used to determine the position and outline of the spots on the target region in the original image based on the point cloud dataset and the original image; The generation module is used to generate a color mode image based on the original image; The second determining module is used to determine the position and outline of the spots on the target area in the color mode image based on the position and outline of the spots on the target area in the original image. The third determining module is used to determine the color information of the spots on the target part in the color mode image based on the position and outline of the spots on the target part in the color mode image; The fourth determining module is used to determine the category of the spots on the target area in the original image based on the color information of the spots on the target area in the color mode image; The laser removal module is used to determine laser-removable spots among the spots on the target area in the original image according to the category of spots on the target area, obtain calibration parameters, and control the laser emitter to remove the laser-removable spots on the target area according to the calibration parameters. The calibration parameters are the conversion coefficient between the position of the laser emitter and the position of the spots on the target area.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images