Target position identification method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a target position identification method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: determining positive sample points located in a first area and negative sample points located in a second area according to a target image by using a reinforcement learning network; determining an initial mask according to the target image, the positive sample point and the negative sample point by using an interactive segmentation network obtained by pre-training, and determining a target position according to the initial mask; performing depth sampling on the target image along a first direction of the target position in the first area to determine a depth sampling result; determining a depth boundary point in a first direction along the target position according to the depth sampling result, and determining a newly added positive sample point and a newly added negative sample point in an area defined by the depth boundary point according to a preset distance; and determining a target mask according to the positive sample point, the negative sample point, the newly added positive sample point and the newly added negative sample point by using the interactive segmentation network, and determining a target area according to the target mask.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a target location recognition method, device, electronic device, storage medium and program product. Background Art

[0002] In the related art, deep learning methods are usually used to collect a large amount of human body data and mark the exposed abdomen position to train an end-to-end abdomen segmentation network, and the acquired image is input into the abdomen segmentation network to output the abdomen mask. However, this solution requires a larger model to achieve a higher learning ability and requires a larger amount of data for training. Therefore, there is a problem of over-reliance on the amount of data, and the recognition accuracy is low when the amount of data is small. Summary of the invention

[0003] In view of this, the purpose of the present application is to provide a target location identification method, device, electronic device, storage medium and program product.

[0004] Based on the above objectives, in a first aspect, the present application provides a method for identifying a target location, the method comprising:

[0005] Determine, by using a reinforcement learning network, positive sample points located in the first region and negative sample points located in the second region according to the target image;

[0006] Determine an initial mask according to the target image, positive sample points and negative sample points using a pre-trained interactive segmentation network, and determine the target position according to the initial mask; wherein the positive sample points are located inside the initial mask, and the negative sample points are located outside the initial mask;

[0007] Performing depth sampling on the target image along a first direction of the target position within the first area to determine a depth sampling result;

[0008] Determine a depth boundary point in a first direction along the target position according to the depth sampling result, and determine a newly added positive sample point and a newly added negative sample point according to a preset distance in an area surrounded by the depth boundary point;

[0009] The interactive segmentation network is used to determine a target mask according to the positive sample points, the negative sample points, the newly added positive sample points and the newly added negative sample points, and a target area is determined according to the target mask; wherein the target mask is used to characterize at least a portion of the first area and the target position.

[0010] In a second aspect, the present application provides a target position identification device, the device comprising:

[0011] A first determination module is configured to determine positive sample points located in the first area and negative sample points located in the second area according to the target image using a reinforcement learning network;

[0012] A second determination module is configured to determine an initial mask according to the target image, the positive sample points and the negative sample points using a pre-trained interactive segmentation network, and determine the target position according to the initial mask; wherein the positive sample points are located inside the initial mask, and the negative sample points are located outside the initial mask;

[0013] a sampling module, configured to perform depth sampling on the target image along a first direction of the target position within the first area to determine a depth sampling result;

[0014] A third determination module is configured to determine a depth boundary point along a first direction of the target position according to the depth sampling result, and determine a newly added positive sample point and a newly added negative sample point according to a preset distance in an area surrounded by the depth boundary point;

[0015] The fourth determination module is configured to use the interactive segmentation network to determine a target mask according to the positive sample points, the negative sample points, the newly added positive sample points and the newly added negative sample points, and determine a target area according to the target mask; wherein the target mask is used to characterize at least a portion of the first area and the target position.

[0016] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the target location identification method as described in the first aspect is implemented.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, wherein the computer instructions are used to enable a computer to execute the target location identification method as described in the first aspect.

[0018] In a fifth aspect, the present application provides a computer program product, comprising computer program instructions, which, when executed on a computer, enable the computer to execute the target location identification method as described in the first aspect.

[0019] As can be seen from the above, the present application provides a method, device, electronic device, storage medium and program product for identifying a target position, which uses a reinforcement learning network to determine positive sample points located in a first area and negative sample points located in a second area according to a target image; uses a pre-trained interactive segmentation network to determine an initial mask according to the target image, positive sample points and negative sample points, and determines the target position according to the initial mask; wherein the positive sample points are located inside the initial mask, and the negative sample points are located outside the initial mask; performs depth sampling on the target image along a first direction of the target position in the first area to determine a depth sampling result; determines depth boundary points along the first direction of the target position according to the depth sampling result, and determines newly added positive sample points and newly added negative sample points according to a preset distance in the area surrounded by the depth boundary points; uses the interactive segmentation network to determine a target mask according to the positive sample points, negative sample points, newly added positive sample points and newly added negative sample points, and determines the target area according to the target mask; wherein the target mask is used to characterize at least part of the first area and the target position. The present application only needs to use a pre-trained interactive segmentation network to learn the first region, such as the abdomen, and the second region, such as the area where the clothes are located, and represent them with positive and negative sample points to obtain accurate segmented regions, and use less self-collected exposed abdomen data and a lightweight network to achieve recognition and determination of the target position, while achieving a high degree of accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A schematic diagram of an exemplary flow chart of a target location identification method provided in an embodiment of the present application is shown.

[0022] Figure 2 An exemplary schematic diagram of positive sample points and negative sample points in a target image according to an embodiment of the present application is shown.

[0023] Figure 3 An exemplary schematic diagram of an initial mask in an embodiment according to the present application is shown.

[0024] Figure 4 An exemplary schematic diagram of a target position in an embodiment according to the present application is shown.

[0025] Figure 5An exemplary schematic diagram of sampling lines in a target image during depth sampling according to an embodiment of the present application is shown.

[0026] Figure 6 An exemplary schematic diagram of newly added positive sample points and newly added negative sample points in a target image according to an embodiment of the present application is shown.

[0027] Figure 7 An exemplary schematic diagram of a target mask according to an embodiment of the present application is shown.

[0028] Figure 8 An exemplary schematic diagram of a target area according to an embodiment of the present application is shown.

[0029] Fig. 9 A schematic diagram of an exemplary structure of a target position identification device provided in an embodiment of the present application is shown.

[0030] Fig.10 A schematic diagram of an exemplary structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0032] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0033] As described in the background technology section, in the related art, a deep learning method is usually used to collect a large amount of human body data and mark the exposed abdomen position to train an end-to-end abdomen segmentation network, and the acquired image is input into the abdomen segmentation network to output an abdomen mask.

[0034] The inventors' research found that this solution requires a larger model to achieve a higher learning ability and a larger amount of data for training. Therefore, there is a problem of over-reliance on the amount of data, and this results in a lower recognition accuracy when the amount of data is smaller.

[0035] In addition, there is another method in the related art which is based on a traditional algorithm and directly solves the area where the skin is located from the image according to the difference between the human skin color and the environment.

[0036] The inventors found through research that the logical rules of this method are relatively complex and limited, and involve some thresholds, and cannot be applied to people of all skin colors.

[0037] For this reason, the present application provides a method, device, electronic device, storage medium and program product for identifying a target position, which uses a reinforcement learning network to determine positive sample points located in a first area and negative sample points located in a second area according to a target image; uses a pre-trained interactive segmentation network to determine an initial mask according to the target image, positive sample points and negative sample points, and determines the target position according to the initial mask; wherein the positive sample points are located inside the initial mask, and the negative sample points are located outside the initial mask; performs depth sampling on the target image along a first direction of the target position in the first area to determine a depth sampling result; determines depth boundary points along the first direction of the target position according to the depth sampling result, and determines newly added positive sample points and newly added negative sample points according to a preset distance in the area surrounded by the depth boundary points; uses the interactive segmentation network to determine a target mask according to the positive sample points, negative sample points, newly added positive sample points and newly added negative sample points, and determines a target area according to the target mask; wherein the target mask is used to characterize at least part of the first area and the target position. The present application only needs to use a pre-trained interactive segmentation network to learn the first region, such as the abdomen, and the second region, such as the area where the clothes are located, and represent them with positive and negative sample points to obtain accurate segmented regions, and use less self-collected exposed abdomen data and a lightweight network to achieve recognition and determination of the target position, while achieving a high degree of accuracy.

[0038] In some specific application scenarios, the target location recognition method of the present application can be applied to various systems involving target location recognition, which can be run through a PC or a mobile terminal such as a mobile phone or tablet computer.

[0039] In some specific application scenarios, the target location identification method of the present application can be directly applied to run locally or in a cloud server. When running in a cloud server, the acquired data to be processed is sent to the cloud server via the network, and the server processes the data to be processed using the target location identification method of the present application, and sends the processing results to the local computer via the network.

[0040] The following describes the target location recognition method according to the exemplary embodiment of the present application in conjunction with a specific application scenario. It should be noted that the above application scenario is only shown to facilitate understanding of the spirit and principle of the present application, and the implementation of the present application is not limited in this regard. On the contrary, the implementation of the present application can be applied to any applicable scenario.

[0041] The target location recognition method provided in the embodiment of the present application is described in detail below through specific examples.

[0042] Figure 1 A schematic diagram of an exemplary flow chart of a target location identification method provided in an embodiment of the present application is shown.

[0043] refer to Figure 1 , a target location recognition method provided by an embodiment of the present application, the method specifically comprises the following steps:

[0044] S102: Determine, by using a reinforcement learning network, positive sample points located in the first area and negative sample points located in the second area according to the target image.

[0045] S104: using a pre-trained interactive segmentation network to determine an initial mask according to the target image, positive sample points and negative sample points, and determining a target position according to the initial mask; wherein the positive sample points are located inside the initial mask, and the negative sample points are located outside the initial mask.

[0046] S106: Performing depth sampling on the target image along a first direction of the target position within the first area to determine a depth sampling result.

[0047] S108: Determine depth boundary points along a first direction of the target position according to the depth sampling result, and determine new positive sample points and new negative sample points according to a preset distance in an area surrounded by the depth boundary points.

[0048] S110: Determine a target mask according to the positive sample points, the negative sample points, the newly added positive sample points and the newly added negative sample points using the interactive segmentation network, and determine a target area according to the target mask; wherein the target mask is used to characterize at least a portion of the first area and the target position.

[0049] Figure 2 An exemplary schematic diagram of positive sample points and negative sample points in a target image according to an embodiment of the present application is shown.

[0050] In some embodiments, reference Figure 2 , the target image can be obtained by shooting a human body image. For example, when massaging a human body, it is necessary to identify the massaged part, taking the abdomen as an example. Figure 2 The middle abdomen is the exposed part of the image, and the other parts of the image include interference images such as the environment image and clothing image. However, people can distinguish the abdomen image and other interference images, but if it is executed by the terminal, it needs to recognize the input target image to identify the target area that needs massage.

[0051] It should be noted that the target image may be an RGB image of a human body obtained by shooting at a first viewing angle, wherein the first viewing angle may be a bird's-eye view.

[0052] Furthermore, a reinforcement learning network can be used to determine positive sample points and negative sample points according to the target image, wherein the positive sample points are located in a first area, such as an exposed abdomen area, and the negative sample points are located in a second area, such as an area other than the first area in the target image. Figure 2 The first area is area A, the second area is area B, the first area contains 3 positive sample points, and the second area contains 12 negative sample points.

[0053] It should be noted that the training process of the reinforcement learning network may include: collecting multiple human images with a first region exposed, and marking the first region with a true mask in the human image; inputting the human image into the reinforcement learning network to be trained to obtain multiple sample points and positive and negative information of each sample point; determining a predicted mask according to the human image, multiple sample points and the positive and negative information of each sample point; determining whether the intersection-and-union ratio between the real mask and the predicted mask reaches a preset threshold; in response to the intersection-and-union ratio between the real mask and the predicted mask reaching the preset threshold, the trained reinforcement learning network is obtained. For example, a number of human images with exposed abdomens are collected, and the abdomen region is masked and marked in the human image to determine the true mask. The human image is input into the reinforcement learning network to be trained, and a sample point and the positive and negative information of the sample point can be obtained. When calculating the loss during the training process, the human image, the sample point and the positive and negative information can be input into the interactive segmentation model to obtain a predicted mask, and the real mask and the predicted mask are compared. The predicted mask and the human image are input into the reinforcement learning model again, and another sample point will be output. And so on to determine all the sample points.

[0054] It should be noted that the reinforcement learning network can adopt a PP0 network, in which the network reward (ie, reward) can be set to the difference between the intersection and union ratio of the predicted mask and the true mask obtained by the current prediction and the intersection and union ratio of the predicted mask and the true mask obtained by the previous prediction.

[0055] Figure 3 An exemplary schematic diagram of an initial mask in an embodiment according to the present application is shown.

[0056] In some embodiments, reference Figure 3 After determining the positive sample points and the negative sample points, the pre-trained interactive segmentation network can be used to determine the initial mask according to the target image and the positive and negative sample points. For example, the initial mask can be a rough abdominal area mask, which ensures that the positive sample points are all within the initial mask and the negative sample points are all outside the initial mask.

[0057] It should be noted that the training process of the interactive segmentation network may include: obtaining a segmentation data set from a public data set; wherein the segmentation data set includes: a training image, a training positive sample point located in a first area of ​​the training image, and a training negative sample point located in a second area of ​​the training image, and the training image is a human body image taken from a first perspective; the segmentation data set is input into the interactive segmentation network to be trained to output a training mask; in response to determining that the training positive sample point is located inside the training mask and the training negative sample point is located outside the training mask, the trained interactive segmentation network is obtained. The first perspective may be a bird's-eye view to ensure that the image used in the training process is the same as the shooting perspective of the target image. For example, the interactive segmentation network in the present application may use HRNet and OCR as the backbone network, and the loss function loss adopts focal loss.

[0058] Figure 4 An exemplary schematic diagram of a target position in an embodiment according to the present application is shown.

[0059] Further, refer to Figure 4 , the target position can be determined according to the initial mask, for example, the target position can be the position of the belly button in the abdomen. Specifically, the pre-trained feature detection network can be used to determine the target position within the first area according to the initial mask, and the feature detection network can be trained based on the human body image of the exposed first area, the target position, and the mapping relationship between the human body image and the target position. For example, the training set may include a human body image with an exposed abdomen, the position of the belly button marked in the human body image, and the corresponding relationship between the two, and the training is performed according to the marking, so that the feature detection network can determine the target position according to the input initial mask.

[0060] In some embodiments, a template matching method can be used to collect and save many images of belly buttons in advance, and then slide the window operation on the image one by one and match it. If the similarity is high, it is considered that this is the location of the belly button. Specifically, multiple template images marked with the target position can be collected; the template image is used as a sliding window to slide along the initial mask, and it is determined whether the similarity between the initial mask in the sliding window and the target position marked in the template image reaches a preset threshold; in response to the similarity between the initial mask in the sliding window and the target position marked in the template image reaching a preset threshold, the position in the initial mask corresponding to the sliding window at the current moment is used as the target position.

[0061] Figure 5 An exemplary schematic diagram of sampling lines in a target image during depth sampling according to an embodiment of the present application is shown.

[0062] In some embodiments, the target image may be depth sampled within the first region along the first direction of the target position to determine a depth sampling result. Figure 5 , multiple sampling lines can be set along the first direction of the target position, and depth sampling is performed on the target image overlapping with the sampling lines along the sampling lines to determine the depth value of the first area and the depth value of the second area. Specifically, three straight lines across the abdomen can be drawn and sampling is performed on the straight lines, wherein the first area, that is, the abdomen area, at least partially overlaps with the sampling lines, and the second area also at least partially overlaps with the sampling lines. The first direction can be the left-right direction along the abdomen area.

[0063] Furthermore, the depth boundary detection model acquired in advance can be used to determine the boundary points of the sampling line in the first area according to the depth value of the first area and the depth value of the second area, and the boundary points of the sampling line in the first area are used as the depth boundary points along the first direction of the target position. Wherein, 100 points can be collected on each straight line, and each straight line obtains an array of [100,1], which is used to represent the depth value.

[0064] It should be noted that the training process of the depth boundary detection model may include: collecting data of several naked abdomens of human bodies and marking the abdomen mask, and then performing depth sampling in a horizontal straight line in the abdomen area in the second direction (for example, along the longitudinal direction of the abdomen area), and the sampling includes the depth values ​​of the abdomen and both sides (bed surface or hand). The model input is the depth sampling result, that is, the depth value of the first area and the depth value of the second area, and the output is the position of the left and right boundary points on the current sampling line. The loss function of the model can use MSE loss.

[0065] Figure 6 An exemplary schematic diagram of newly added positive sample points and newly added negative sample points in a target image according to an embodiment of the present application is shown.

[0066] Further, refer to Figure 6 , within the area surrounded by the depth boundary points, determine a point selection area with the target position as the center point and the preset distance as the point selection radius; select a preset number of sample points within the point selection area as the newly added positive sample points, and select a preset number of sample points outside the point selection area and within the area surrounded by the depth boundary points as the newly added negative sample points. For example, if the belly button position and boundary points are known, then the positions near the boundary points that are closer to the belly button are taken as newly added positive sample points, and the positions that are farther away from the belly button are taken as newly added negative sample points. Figure 6 In the method, the number of positive sample points and newly added positive sample points can be increased to 5, and the number of negative sample points and newly added negative sample points can be increased to 15. The sample data is increased without increasing the target image.

[0067] Figure 7 An exemplary schematic diagram of a target mask according to an embodiment of the present application is shown.

[0068] In some embodiments, the interactive segmentation network can be used to determine a target mask based on the positive sample points, the negative sample points, the newly added positive sample points, and the newly added negative sample points. The target mask can be understood as a mask containing the target position with higher accuracy. The target mask is used to characterize at least part of the first area and the target position. For example, referring to Figure 7 , the target mask may be a more accurate abdominal region mask including the location of the belly button.

[0069] Figure 8 An exemplary schematic diagram of a target area according to an embodiment of the present application is shown.

[0070] Furthermore, the target area can be determined according to the target mask, that is, the reference Figure 8 In the process, according to the superposition between the target mask and the target image, the target area corresponding to the target mask in the target image can be determined. The target area can be, for example, a service area including the location of the belly button. When massaging the target area, the terminal can determine the target area through the target image and massage the target area to prevent the robot from massaging areas other than the target area, such as clothes, arms, or bed surfaces.

[0071] As can be seen from the above, the present application provides a method, device, electronic device, storage medium and program product for identifying a target position, which uses a reinforcement learning network to determine positive sample points located in a first area and negative sample points located in a second area according to a target image; uses a pre-trained interactive segmentation network to determine an initial mask according to the target image, positive sample points and negative sample points, and determines the target position according to the initial mask; wherein the positive sample points are located inside the initial mask, and the negative sample points are located outside the initial mask; performs depth sampling on the target image along a first direction of the target position in the first area to determine a depth sampling result; determines depth boundary points along the first direction of the target position according to the depth sampling result, and determines newly added positive sample points and newly added negative sample points according to a preset distance in the area surrounded by the depth boundary points; uses the interactive segmentation network to determine a target mask according to the positive sample points, negative sample points, newly added positive sample points and newly added negative sample points, and determines the target area according to the target mask; wherein the target mask is used to characterize at least part of the first area and the target position. The present application only needs to use a pre-trained interactive segmentation network to learn the first region, such as the abdomen, and the second region, such as the area where the clothes are located, and represent them with positive and negative sample points to obtain accurate segmented regions, and use less self-collected exposed abdomen data and a lightweight network to achieve recognition and determination of the target position, while achieving a high degree of accuracy.

[0072] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0073] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0074] Fig. 9 A schematic diagram of an exemplary structure of a target position identification device provided in an embodiment of the present application is shown.

[0075] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a target position identification device.

[0076] refer to Fig. 9 , the target position identification device includes: a first determination module, a second determination module, a sampling module, a third determination module and a fourth determination module; wherein,

[0077] A first determination module is configured to determine positive sample points located in the first area and negative sample points located in the second area according to the target image using a reinforcement learning network;

[0078] A second determination module is configured to determine an initial mask according to the target image, the positive sample points and the negative sample points using a pre-trained interactive segmentation network, and determine the target position according to the initial mask; wherein the positive sample points are located inside the initial mask, and the negative sample points are located outside the initial mask;

[0079] a sampling module, configured to perform depth sampling on the target image along a first direction of the target position within the first area to determine a depth sampling result;

[0080] A third determination module is configured to determine a depth boundary point along a first direction of the target position according to the depth sampling result, and determine a newly added positive sample point and a newly added negative sample point according to a preset distance in an area surrounded by the depth boundary point;

[0081] The fourth determination module is configured to use the interactive segmentation network to determine a target mask according to the positive sample points, the negative sample points, the newly added positive sample points and the newly added negative sample points, and determine a target area according to the target mask; wherein the target mask is used to characterize at least a portion of the first area and the target position.

[0082] In a possible implementation, the device further includes: a training module;

[0083] The training module is further configured to:

[0084] Collecting a plurality of human body images with a first region exposed, and marking a true mask for the first region in the human body images;

[0085] Inputting the human body image into a reinforcement learning network to be trained to obtain a plurality of sample points and positive and negative information of each sample point;

[0086] Determining a prediction mask according to the human body image, a plurality of sample points, and positive and negative information of each sample point;

[0087] Determining whether an intersection-over-union ratio between the real mask and the predicted mask reaches a preset threshold;

[0088] In response to the intersection-and-union ratio between the real mask and the predicted mask reaching a preset threshold, the trained reinforcement learning network is obtained.

[0089] In a possible implementation, the training module is further configured to:

[0090] Obtain a segmented data set from a public data set; wherein the segmented data set includes: a training image, training positive sample points located in a first area of ​​the training image, and training negative sample points located in a second area of ​​the training image, wherein the training image is a human body image captured from a first perspective;

[0091] Inputting the segmentation data set into an interactive segmentation network to be trained to output a training mask;

[0092] In response to determining that the training positive sample points are located inside the training mask and the training negative sample points are located outside the training mask, the trained interactive segmentation network is obtained.

[0093] In a possible implementation manner, the second determining module is further configured to:

[0094] The target position within the first area is determined according to the initial mask using a pre-trained feature detection network; wherein the feature detection network is trained based on a human body image of the exposed first area, the target position, and a mapping relationship between the human body image and the target position.

[0095] In a possible implementation, the device further includes: a collection module;

[0096] The acquisition module is further configured as follows:

[0097] A plurality of template images with the target positions marked thereon are collected.

[0098] In a possible implementation manner, the second determining module is further configured to:

[0099] Slide the template image as a sliding window along the initial mask, and determine whether the similarity between the initial mask in the sliding window and the target position marked in the template image reaches a preset threshold;

[0100] In response to the similarity between the initial mask in the sliding window and the target position marked in the template image reaching a preset threshold, the position in the initial mask corresponding to the sliding window at the current moment is used as the target position.

[0101] In a possible implementation, the depth sampling result includes: a depth value of the first area and a depth value of the second area;

[0102] The sampling module is further configured to:

[0103] A plurality of sampling lines are arranged along a first direction of the target position; wherein the first area at least partially overlaps with the sampling line, and the second area at least partially overlaps with the sampling line;

[0104] Depth sampling is performed on the target image overlapping the sampling line along the sampling line to determine a depth value of the first area and a depth value of the second area.

[0105] In a possible implementation, the sampling module is further configured to:

[0106] Determine the boundary point of the sampling line in the first area according to the depth value of the first area and the depth value of the second area using the pre-acquired depth boundary detection model

[0107] In a possible implementation manner, the third determining module is further configured to:

[0108] Boundary points of the sampling line in the first area are used as depth boundary points along a first direction of the target position.

[0109] In a possible implementation manner, the third determining module is further configured to:

[0110] In the area surrounded by the depth boundary points, determining a point selection area with the target position as the center point and the preset distance as the point selection radius;

[0111] A preset number of sample points are selected in the selected point area as the newly added positive sample points, and a preset number of sample points are selected outside the selected point area and in an area surrounded by the depth boundary points as the newly added negative sample points.

[0112] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0113] The device of the above embodiment is used to implement the corresponding target position recognition method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0114] Fig.10 A schematic diagram of an exemplary structure of an electronic device provided in an embodiment of the present application is shown.

[0115] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the target position identification method described in any of the above embodiments is implemented. Fig.10 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0116] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0117] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0118] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0119] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0120] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0121] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0122] The electronic device of the above embodiment is used to implement the corresponding target position recognition method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0123] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the target position identification method described in any of the above embodiments.

[0124] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0125] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the target position recognition method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0126] Based on the same inventive concept, corresponding to the target location identification method described in any of the above embodiments, the present disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer so that the computer and / or the processor execute the target location identification method. Corresponding to the execution subject corresponding to each step in each embodiment of the target location identification method, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0127] The computer program product of the above embodiment is used to enable the computer and / or the processor to execute the target position identification method described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0128] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0129] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0130] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0131] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A target area recognition method, characterized in that: The method comprises: Determine, by using a reinforcement learning network, positive sample points located in the first region and negative sample points located in the second region according to the target image; Determine an initial mask according to the target image, positive sample points and negative sample points using a pre-trained interactive segmentation network, and determine the target position according to the initial mask; wherein the positive sample points are located inside the initial mask, and the negative sample points are located outside the initial mask; Performing depth sampling on the target image along a first direction of the target position within the first area to determine a depth sampling result; Determine a depth boundary point in a first direction along the target position according to the depth sampling result, and determine a newly added positive sample point and a newly added negative sample point according to a preset distance in an area surrounded by the depth boundary point; The interactive segmentation network is used to determine a target mask according to the positive sample points, the negative sample points, the newly added positive sample points and the newly added negative sample points, and a target area is determined according to the target mask; wherein the target mask is used to characterize at least a portion of the first area and the target position.

2. The method according to claim 1, characterized in that The training process of the reinforcement learning network includes: Collecting a plurality of human body images with a first region exposed, and marking a true mask for the first region in the human body images; Inputting the human body image into a reinforcement learning network to be trained to obtain a plurality of sample points and positive and negative information of each sample point; Determining a prediction mask according to the human body image, a plurality of sample points, and positive and negative information of each sample point; Determining whether an intersection-over-union ratio between the real mask and the predicted mask reaches a preset threshold; In response to the intersection-and-union ratio between the real mask and the predicted mask reaching a preset threshold, the trained reinforcement learning network is obtained.

3. The method according to claim 1, characterized in that The training process of the interactive segmentation network includes: Obtain a segmented data set from a public data set; wherein the segmented data set includes: a training image, training positive sample points located in a first area of ​​the training image, and training negative sample points located in a second area of ​​the training image, wherein the training image is a human body image captured from a first perspective; Inputting the segmentation data set into an interactive segmentation network to be trained to output a training mask; In response to determining that the training positive sample points are located inside the training mask and the training negative sample points are located outside the training mask, the trained interactive segmentation network is obtained.

4. The method according to claim 1, characterized in that: The determining the target position according to the initial mask comprises: The target position within the first area is determined according to the initial mask using a pre-trained feature detection network; wherein the feature detection network is trained based on a human body image of the exposed first area, the target position, and a mapping relationship between the human body image and the target position.

5. The method according to claim 1, characterized in that Before determining the target position according to the initial mask, the method further includes: Acquiring a plurality of template images marked with the target positions; The determining the target position according to the initial mask comprises: Slide the template image as a sliding window along the initial mask, and determine whether the similarity between the initial mask in the sliding window and the target position marked in the template image reaches a preset threshold; In response to the similarity between the initial mask in the sliding window and the target position marked in the template image reaching a preset threshold, the position in the initial mask corresponding to the sliding window at the current moment is used as the target position.

6. The method according to claim 1, characterized in that The depth sampling result includes: a depth value of the first area and a depth value of the second area; The performing depth sampling on the target image along a first direction of the target position within the first area to determine a depth sampling result includes: A plurality of sampling lines are arranged along a first direction of the target position; wherein the first area at least partially overlaps with the sampling line, and the second area at least partially overlaps with the sampling line; Depth sampling is performed on the target image overlapping the sampling line along the sampling line to determine a depth value of the first area and a depth value of the second area.

7. The method according to claim 1, characterized in that After performing depth sampling on the target image overlapping the sampling line along the sampling line to determine the depth value of the first area and the depth value of the second area, the method further includes: Determine a boundary point of the sampling line in the first area according to the depth value of the first area and the depth value of the second area using a pre-acquired depth boundary detection model; The step of determining a depth boundary point in a first direction along the target position according to the depth sampling result includes: Boundary points of the sampling line in the first area are used as depth boundary points along a first direction of the target position.

8. The method according to claim 1, characterized in that The determining of the newly added positive sample points and the newly added negative sample points according to the preset distance in the area surrounded by the depth boundary points includes: In the area surrounded by the depth boundary points, determining a point selection area with the target position as the center point and the preset distance as the point selection radius; A preset number of sample points are selected in the point selection area as the newly added positive sample points, and a preset number of sample points are selected outside the point selection area and in the area surrounded by the depth boundary points as the newly added negative sample points.

9. A target position identification device, characterized in that: The device comprises: A first determination module is configured to determine positive sample points located in the first area and negative sample points located in the second area according to the target image using a reinforcement learning network; A second determination module is configured to determine an initial mask according to the target image, the positive sample points and the negative sample points using a pre-trained interactive segmentation network, and determine the target position according to the initial mask; wherein the positive sample points are located inside the initial mask, and the negative sample points are located outside the initial mask; a sampling module, configured to perform depth sampling on the target image along a first direction of the target position within the first area to determine a depth sampling result; A third determination module is configured to determine a depth boundary point along a first direction of the target position according to the depth sampling result, and determine a newly added positive sample point and a newly added negative sample point according to a preset distance in an area surrounded by the depth boundary point; The fourth determination module is configured to use the interactive segmentation network to determine a target mask according to the positive sample points, the negative sample points, the newly added positive sample points and the newly added negative sample points, and determine a target area according to the target mask; wherein the target mask is used to characterize at least a portion of the first area and the target position.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to implement the method according to any one of claims 1 to 8.

12. A computer program product comprising computer program instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 8.

Images