Reachable space data enhancement method and model training method based on ground processing

Abstract

The embodiment of the present application discloses a reachable space data enhancement method based on ground processing and a model training method, the method comprising: obtaining a first data set, the first data set comprising a plurality of first images respectively annotated with reachable spaces; based on the number of first images in the first data set, obtaining a second data set comprising a plurality of second images, each second image respectively annotated with reachable spaces; selecting a first target image from the first data set, and selecting a second target image from the second data set; based on the ground data corresponding to the reachable space annotated in the second target image, processing the ground data corresponding to the reachable space annotated in the first target image to obtain the target image. By adopting the above method of the present application, a small amount of special-shaped data is used to annotate a large amount of ordinary data, which reduces the cost of obtaining a large amount of annotated images, thereby increasing the types of samples for training the model.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing, and more specifically, to a achievable space data enhancement method and a model training method based on ground processing. Background Art

[0002] At present, supervised training data is a very important resource when using machine learning to train models. In the related art, in order to obtain a model with higher accuracy, a large amount of sample data is usually used to train the model. Among them, in the task of using the model to detect the reachable space, the ground conditions corresponding to the reachable space are often changeable and complex. If a large amount of manual annotation is performed on each type of ground to obtain a large amount of sample data for model training, the cost of model training will be greatly increased. Therefore, in the related art, there is a problem of high cost in obtaining a large amount of data labeled with reachable space. Summary of the invention

[0003] The embodiments of the present application provide a achievable space data enhancement method and a model training method based on ground processing to improve the above-mentioned problems.

[0004] In a first aspect, an embodiment of the present application provides a method for enhancing reachable space data based on ground processing, the method comprising: obtaining a first data set, wherein the first data set includes multiple first images, each of which is respectively marked with a reachable space; based on the number of the first images in the first data set, obtaining a second data set including multiple second images, each of which is respectively marked with a reachable space, wherein the ground data corresponding to the reachable space marked in the second image is different from the ground data corresponding to the reachable space marked in the first image; selecting a first target image from the first data set, and selecting a second target image from the second data set; processing the ground data corresponding to the reachable space marked in the first target image based on the ground data corresponding to the reachable space marked in the second target image to obtain a target image.

[0005] In a second aspect, an embodiment of the present application further provides a model training method, the method comprising: obtaining an input sample image set, the sample image set comprising a plurality of target images obtained based on the method described in the first aspect above; and training a target model based on the target images.

[0006] In a third aspect, an embodiment of the present application also provides a device for enhancing data of a reachable space based on ground processing, the device comprising: a first data set acquisition unit, a second data set acquisition unit, a target image selection unit and a data enhancement unit. Specifically, the first data set acquisition unit is used to acquire a first data set, wherein the first data set includes a plurality of first images, each of which is respectively annotated with a reachable space; the second data set acquisition unit is used to acquire a second data set including a plurality of second images based on the number of the first images in the first data set, each of which is respectively annotated with a reachable space, wherein the ground data corresponding to the reachable space annotated in the second image is different from the ground data corresponding to the reachable space annotated in the first image; the target image selection unit is used to select a first target image from the first data set, and to select a second target image from the second data set; the data enhancement unit is used to process the ground data corresponding to the reachable space annotated in the first target image based on the ground data corresponding to the reachable space annotated in the second target image, so as to obtain a target image.

[0007] In a fourth aspect, an embodiment of the present application further provides a model training device, which includes: a sample image set acquisition unit and a model training unit. The sample image set acquisition unit is used to acquire an input sample image set, wherein the sample image set includes multiple target images obtained based on the method described in the first aspect above; and the model training unit is used to train a target model based on the target image.

[0008] In a fifth aspect, an embodiment of the present application further provides an electronic device, comprising: one or more processors, a memory, and one or more applications. The one or more applications are stored in the memory and configured to be executed by the one or more processors, so that the one or more processors implement a reachable space data enhancement method based on ground processing described in any of the above embodiments or a model training method described in any of the above embodiments.

[0009] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a program code is stored. The program code can be called by a processor to execute a method for enhancing achievable space data based on ground processing as described in any of the above embodiments or a model training method as described in any of the above embodiments.

[0010] The technical solution provided by the present application is to obtain a first data set, wherein the first data set includes a plurality of first images, each of which is respectively annotated with a reachable space; based on the number of the first images in the first data set, obtain a second data set including a plurality of second images, each of which is respectively annotated with a reachable space, wherein the ground data corresponding to the reachable space annotated in the second image is different from the ground data corresponding to the reachable space annotated in the first image; select a first target image from the first data set, and select a second target image from the second data set; based on the ground data corresponding to the reachable space annotated in the second target image, process the ground data corresponding to the reachable space annotated in the first target image to obtain a target image. Therefore, the above method of the present application is adopted, and the ground data corresponding to the reachable space in a small number of images annotated with reachable space, i.e., a small number of special-shaped data, is used to process the ground data corresponding to the reachable space in a large number of images annotated with reachable space, i.e., a large number of ordinary data, to obtain a large number of images annotated with reachable space that are different from the ground data corresponding to the reachable space annotated in the first image, and the data type of the reachable space is enhanced based on the ground processing method, thereby increasing the types of samples of the training model, reducing the cost of model training, and improving the accuracy of the trained model. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 A schematic diagram of a process flow of a method for enhancing accessible space data based on ground processing provided in an embodiment of the present application is shown;

[0013] Figure 2 A schematic diagram of a reachable space of annotated images provided by an embodiment of the present application is shown;

[0014] Figure 3 A schematic diagram of a reachable space of annotated images provided by another embodiment of the present application is shown;

[0015] Figure 4 A schematic diagram of a first target image provided by an embodiment of the present application is shown;

[0016] Figure 5 A schematic diagram of a second target image provided by an embodiment of the present application is shown;

[0017] Figure 6A schematic diagram showing an overlap area provided by an embodiment of the present application is shown;

[0018] Figure 7 A schematic diagram of a target image provided by an embodiment of the present application is shown;

[0019] Figure 8 A schematic diagram of a first target image provided by another embodiment of the present application is shown;

[0020] Fig. 9 A schematic diagram of a second target image provided by another embodiment of the present application is shown;

[0021] Fig.10 A schematic diagram showing an overlap area provided by another embodiment of the present application is shown;

[0022] Fig.11 A schematic diagram of a target image provided by another embodiment of the present application is shown;

[0023] Fig.12 A schematic diagram of a flow chart of a model training method provided in one embodiment of the present application is shown;

[0024] Fig.13 A structural block diagram of a reachable space data enhancement device based on ground processing provided by an embodiment of the present application is shown;

[0025] Fig.14 A structural block diagram of a model training device provided by an embodiment of the present application is shown;

[0026] Fig.15 A structural block diagram of an electronic device provided by an embodiment of the present application is shown;

[0027] Fig.16 A structural block diagram of a computer-readable storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0030] At present, when using machine learning to train models, data used to guide model training is a very important resource, but the cost is relatively high. In order to obtain a more accurate model, a large amount of data to guide model training is often required. If a large amount of data to guide model training is obtained manually, the cost of model training will be too high.

[0031] In the autonomous driving technology, there is a task of detecting the accessible space of the vehicle. The accessible space of the vehicle is the ground, and the conditions on the ground are often changeable and complex. To obtain a high-precision accessible space detection model, it is necessary to obtain a large number of different images with marked accessible space and corresponding ground data of the accessible space, and use these images to guide the training of the accessible space detection model. If these images are obtained manually, although these images are logical, using these images to train the model ensures the accuracy of the model, but it will greatly increase the cost of model training; if a model of forged images is used, images with marked accessible space are used to forge images with different marked accessible space with ground data, although a large number of images with marked accessible space are obtained, due to the uncertainty of the neural network in the model of forged images, using these images to train the model will not guarantee the accuracy of the model obtained after training. Exemplarily, based on the adversarial generative network (GAN) forged images, the artificially obtained images are input into the adversarial generative network; the adversarial generative network will generate images similar to the artificially obtained images, but due to the uncontrollable neural network in the adversarial generative network, the images generated by the adversarial generative network may not be logical, so the use of images generated by the adversarial generative network to train the model has certain uncertainty, which will affect the accuracy of the model.

[0032] In order to alleviate the above problems, the inventor of the present application has proposed a reachable space data enhancement method and model training method based on ground processing provided by the present application. By obtaining a first data set, wherein the first data set includes multiple first images, each of which is respectively annotated with a reachable space; based on the number of the first images in the first data set, a second data set including multiple second images is obtained, each of which is respectively annotated with a reachable space, wherein the ground data corresponding to the reachable space annotated in the second image is different from the ground data corresponding to the reachable space annotated in the first image; a first target image is selected from the first data set, and a second target image is selected from the second data set; based on the ground data corresponding to the reachable space annotated in the second target image, the ground data corresponding to the reachable space annotated in the first target image is processed to obtain a target image. Therefore, by adopting the above method of the present application, a large amount of ordinary data is annotated with a small amount of special-shaped data, a large number of target images with logically annotated reachable spaces and reachable spaces corresponding to different ground types are obtained, which reduces the cost of obtaining a large number of images with annotated reachable spaces and different ground data corresponding to the reachable spaces, and at the same time, the target image is used to train the model, which reduces the cost of model training and improves the accuracy of the trained model.

[0033] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0034] See also Figure 1 , which shows a flow chart of a method for enhancing reachable space data based on ground processing provided in an embodiment of the present application. This embodiment describes the step flow on the terminal device side, and the method may include steps S110 to S140.

[0035] Step S110: Acquire a first data set, wherein the first data set includes a plurality of first images, and each first image is respectively annotated with a reachable space.

[0036] In an embodiment of the present application, the terminal device obtains the first data set from an associated cloud or electronic device through wireless communication technology (such as WiFi, Bluetooth, ZigbEE, etc.), or from an associated electronic device through a serial communication interface (such as SPI), or it can be an image set collected by the terminal device through components such as cameras, lidars, and visual sensors.

[0037] In the embodiments of the present application, the accessible space may be an area where a vehicle can drive, an area where a person can walk, an area where a mobile robot can move, etc.; the accessible space may be various types of ground areas (such as cement ground, water-logged ground, reflective ground, sandy ground, etc.). For example, when a vehicle starts from the current position and drives to a wall and stops, the ground area without obstacles in this driving route is the accessible space.

[0038] It should be understood that the area outside the accessible space in the image is the non-accessible space, and the accessible space in the image is the ground. The ground data corresponding to the accessible space in the image can be cement ground data, waterlogged ground data, reflective ground data, sandy ground data, etc. in the image.

[0039] In some embodiments, the way to mark the accessible space in the image can be to use a line to mark the intersection line between the ground and the obstacle in the image (for example, if the obstacle is a wall, the marked line can be the intersection line between the wall and the ground in the image; if the obstacle is a person, the marked line can be the intersection line between the person and the ground in the image), that is, to use a line to mark the intersection line between the accessible space and the non-accessible space; the area of ​​the accessible space in the image can also be drawn using a polygon, and the area within the polygon is the accessible space. For example, please refer to Figure 2 When a label line is used to mark the reachable space in an image, the image area corresponding to the label line in the image is the reachable space; see Figure 3 ,When the reachable space in an image is annotated with a labeled polygon, the area within the polygon in the image is the reachable space.

[0040] In the embodiment of the present application, the first data set includes a plurality of first images, and each first image is respectively annotated with a reachable space.

[0041] Among them, the type of ground corresponding to the reachable space in each first image in the first data set is the same; the method of marking the reachable space in each first image in the first data set is the same; the number of first images in the first data set can be between tens of thousands and millions.

[0042] In an optional implementation provided in an embodiment of the present application, considering that most types of ground in daily life are cement ground, it is relatively low cost to obtain the ground data with accessible space marked and corresponding to the accessible space as cement ground data, and a large number of images with accessible space marked and corresponding to the accessible space as cement ground data can be obtained as the first data set, and the ground data corresponding to the accessible space in each first image in the first data set is cement ground data. Among them, the way to mark the accessible space of the first image can be to use a line to mark the intersection line between the accessible space and the non-accessible space, wherein the area corresponding to the boundary line is the accessible space, and the ground corresponding to the accessible space is the cement ground, or to use a marked polygon to mark the accessible space in the first image, and the area within the polygon in the first image is the accessible space. Specifically, the number of first images in the first data set can be in the order of 1 million (e.g., the first data set includes 1 million first images).

[0043] Step S120: Based on the number of the first images in the first data set, obtain a second data set including multiple second images, each of the second images is annotated with a reachable space, wherein the ground data corresponding to the reachable space annotated in the second image is different from the ground data corresponding to the reachable space annotated in the first image.

[0044] In an embodiment of the present application, the ground data corresponding to the reachable space in each second image in the second data set may be the same or different; the method of marking the reachable space in each second image in the second data set is the same; the reachable space of the second image in the second data set may be various types of ground (such as concrete ground, water-logged ground, reflective ground, sandy ground, etc.).

[0045] In an optional implementation provided in an embodiment of the present application, considering that the cost of acquiring images with accessible spaces marked as cement floors is low in daily life, and the cost of acquiring images with accessible spaces marked as non-cement floors is relatively high, a small number of images with accessible spaces marked as non-cement floors can be acquired as a second data set to reduce the cost of acquiring images with accessible spaces marked as such. Among them, the ground data corresponding to the accessible space in each second image in the second data set is non-cement floor data. Specifically, the number of second images with a small number of accessible spaces marked as non-cement floor data in the second data set is within a numerical range obtained by multiplying a preset selection ratio range by the number of first images in the first data set.

[0046] In the embodiment of the present application, the ground data may include the type of ground and the area of ​​the ground. The type of ground may be different ground types (such as cement ground, water-logged ground, reflective ground, sandy ground, etc.); the area of ​​the ground is the reachable space area, which may include the area and shape of the ground corresponding to the reachable space.

[0047] It should be understood that in a method for enhancing reachable space data based on ground processing provided in an embodiment of the present application, the number of first images in a first data set is greater than the number of second images in a second data set, wherein a large amount of ground data corresponding to the reachable space in the first image with the reachable space marked is different from a small amount of ground data corresponding to the reachable space in the second image with the reachable space marked, that is, the type of ground corresponding to the reachable space in the first image is different from the type of ground corresponding to the reachable space in the second image.

[0048] Among them, the number of second images in the second data set is less than the number of first images in the first data set; the ground data corresponding to the reachable space in the second image is different from the ground data corresponding to the reachable space in the first image; the ground data corresponding to the reachable space in the first image is usually common cement ground, and the ground data corresponding to the reachable space in the second image is non-cement ground, so the ground data corresponding to the reachable space in the second image with the reachable space marked in the second data set can be understood as a small amount of special-shaped data; the ground data corresponding to the reachable space in the first image with the reachable space marked in the first data set can be understood as a large amount of ordinary data.

[0049] Among them, the preset selection ratio range can be pre-stored in the terminal device, or obtained from the associated cloud or electronic device through wireless communication technology, or obtained from the associated electronic device through a serial communication interface; wherein the size of the preset selection ratio range can be set by the user independently, or obtained through third-party experimental data.

[0050] Exemplarily, the preset selection ratio range is 1% to 3%, which is pre-stored in the terminal device through user self-setting; the type of ground corresponding to the accessible space in each first image in the first data set is cement ground; the way to mark the accessible space of the first image is to use a line to mark the intersection line between the accessible space and the non-accessible space, wherein the area corresponding to the intersection line is the accessible space; the number of first images in the first data set is 1 million. Based on the preset selection ratio range and the number of first images in the first data set, a second data set including multiple second images is obtained. Wherein, the type of ground corresponding to the accessible space marked in the second image in the second data set is different from the type of ground corresponding to the accessible space marked in the first image, that is, the type of ground corresponding to the accessible space in each second image in the second data set is a non-cement ground type of ground (such as a bottom surface of accumulated water, a reflective ground, a sandy ground, etc.); the way to mark the accessible space of the second image is to use a line to mark the intersection line between the accessible space and the non-accessible space, wherein the area corresponding to the intersection line in the second image is the accessible space; the number of second images in the second data set is in the range of 10,000 to 30,000 (1 million × 1% to 1 million × 3%).

[0051] Step S130: selecting a first target image from the first data set, and selecting a second target image from the second data set.

[0052] In an embodiment of the present application, selecting the first target image from the first data set may be randomly selecting a first image from the first data set as the first target image, or selecting a first image with the largest reachable spatial area from the first data set as the first target image, or selecting a first image with the smallest reachable spatial area from the first data set as the first target image, and there is no limitation on this.

[0053] Similarly, selecting the second target image from the second data set may be randomly selecting a second image from the second data set as the second target image, or selecting a second image with the largest reachable spatial area from the second data set as the second target image, or selecting a second image with the smallest reachable spatial area from the second data set as the second target image, and there is no limitation here.

[0054] It should be understood that the method of selecting the first target image from the first data set and the method of selecting the second target image from the second data set may be the same or different.

[0055] Step S140: Processing the ground data corresponding to the reachable space marked in the first target image based on the ground data corresponding to the reachable space marked in the second target image to obtain a target image.

[0056] In an embodiment of the present application, the ground data corresponding to the reachable space marked in the first target image is processed based on the ground data corresponding to the reachable space marked in the second target image to obtain the target image. The target image can be obtained by replacing the ground data corresponding to the reachable space marked in the first target image with the ground data corresponding to the reachable space marked in the second target image; or the target image can be obtained by splicing the ground data corresponding to the reachable space marked in the second target image with the ground data corresponding to the reachable space marked in the first target image.

[0057] Specifically, the ground data corresponding to the reachable space marked in the first target image is used to replace the ground data corresponding to the reachable space marked in the second target image, and the target image obtained can be: if the first area corresponding to the reachable space marked in the first target image is larger than the second area corresponding to the reachable space marked in the second target image, the overlapping area between the first area and the second area is obtained; the ground data of the overlapping area in the second target image is used as the ground data of the first replacement area; the ground data of the second replacement area is selected from the ground data in the overlapping area in the second target image, and the size of the second replacement area is the same as the area in the first area except the overlapping area; the ground data of the first replacement area is spliced ​​with the ground data of the second replacement area to obtain the ground data of the target replacement area; the ground data of the first area in the first target image is replaced with the ground data of the target replacement area to obtain the target image.

[0058] For example, see Figure 4 , select a first target image from the first data set, the first target image is as follows Figure 4 shown; see Figure 5 , select a second target image from the second data set, the second target model is as follows Figure 5 As shown; wherein the first area corresponding to the reachable space marked in the first target image (such as Figure 4 1) is larger than the second region corresponding to the reachable space marked in the second target image (as shown in FIG. Figure 5 2), obtain the overlapping area of ​​the first area and the second area, refer to Figure 6 , the overlap area is as follows Figure 6 4; using the ground data of the overlapping area in the second target image as the ground data of the first replacement area; selecting the ground data of the second replacement area from the ground data of the overlapping area in the second target image, the size of the second replacement area is the same as the area in the first area except the overlapping area (such as Figure 6 3 areas); splicing the ground data of the first replacement area with the ground data of the second replacement area to obtain the ground data of the target replacement area; using the ground data of the target replacement area to replace the ground data of the first area in the first target image, to obtain the target image, see Figure 7 .

[0059] It should be understood that the target image is obtained by replacing the ground data corresponding to the reachable space marked in the first target image with the ground data corresponding to the reachable space marked in the second target image, wherein the image data corresponding to the non-reachable space in the target image is the same as the image data corresponding to the non-reachable space in the first target image; the method of marking the reachable space of the target image is the same as the method of marking the first target image, and the data marking the reachable space of the target image is the same as the data marking the first target image (such as the same line or the same polygon).

[0060] In some embodiments, if the first area corresponding to the reachable space marked in the first target image is not larger than the second area corresponding to the reachable space marked in the second target image, the ground data corresponding to the reachable space marked in the second target image is used to replace the ground data corresponding to the reachable space marked in the first target image to obtain the target image. The target image may be obtained by obtaining the overlapping area between the first area and the second area; using the ground data of the overlapping area in the second target image as the ground data of the target replacement area; and replacing the ground data of the first area in the first target image with the ground data of the target replacement area to obtain the target image.

[0061] For example, see Figure 8 , select a first target image from the first data set, the first target image is as follows Figure 8 shown; see Fig. 9 , select a second target image from the second data set, wherein the second target image is as follows Fig. 9 As shown; wherein the first area corresponding to the reachable space marked in the first target image (such as Figure 8 The area a in the figure is smaller than the second area corresponding to the reachable space marked in the second target image (as shown in Fig. 9 ), obtain the overlapping area of ​​the first area and the second area (as shown in area b in FIG. Fig.10 The ground data of the overlapping area in the second target image is used as the ground data of the target replacement area; the ground data of the first area in the first target image is replaced by the ground data of the target replacement area to obtain the target image. Fig.11 .

[0062] In an optional implementation provided in an embodiment of the present application, considering that the number of second images with marked reachable spaces in the second data set is less than the number of first images with marked reachable spaces in the first data set, in order to reduce the cost of obtaining a large number of images with marked reachable spaces corresponding to ground data corresponding to the reachable spaces in the second images in the second data set, the ground data in the second image with marked reachable spaces in the second data set can be used to repeatedly replace the ground data in the first image with marked reachable spaces in the first data set, so as to obtain a large number of images with logically reachable space areas that are the same as the first image but with different types of ground corresponding to the reachable spaces, thereby enhancing the data type of the reachable space in the image, that is, using a small amount of special-shaped data to annotate a large amount of ordinary data.

[0063] The technical solution of the present application is to obtain a first data set, wherein the first data set includes multiple first images, each of which is respectively annotated with a reachable space; based on the number of first images in the first data set, obtain a second data set including multiple second images, each of which is respectively annotated with a reachable space, wherein the ground data corresponding to the reachable space annotated in the second image is different from the ground data corresponding to the reachable space annotated in the first image; select a first target image from the first data set, and select a second target image from the second data set; based on the ground data corresponding to the reachable space annotated in the second target image, process the ground data corresponding to the reachable space annotated in the first target image to obtain a target image. Therefore, by adopting the above method of the present application, a large amount of ordinary data is annotated with a small amount of special-shaped data, a large amount of target images annotated with reachable space and different ground data corresponding to the reachable space are obtained, the data type of the reachable space is enhanced in a ground-based processing manner, and the cost of annotating the reachable space in the image is reduced.

[0064] See also Fig.12 , which shows a flow chart of a model training method provided in an embodiment of the present application. This embodiment describes the step flow on the terminal device side, and the method may include steps S210 to S220.

[0065] Step S210: obtaining an input sample image set, wherein the sample image set includes a plurality of target images obtained by a reachable space data enhancement method based on ground processing provided in an embodiment of the present application.

[0066] In an embodiment of the present application, the input sample image set can be obtained from an associated cloud or electronic device through wireless communication technology, or obtained from an associated electronic device through a serial communication interface, or obtained by a terminal device using a reachable space data enhancement method based on ground processing provided in an embodiment of the present application to obtain a target image. Each target image is respectively annotated with a reachable space in the image. A large number of target images are obtained through a reachable space data enhancement method based on ground processing provided in an embodiment of the present application, which is low-cost and logical.

[0067] Specifically, for the detailed process of the terminal device applying a terrestrial processing-based reachable space data enhancement method provided in an embodiment of the present application to obtain a target image, please refer to the specific description of steps S110 to S140 above, which will not be repeated here.

[0068] Step S220: training a target model based on the target image.

[0069] In some embodiments, the target model may be a network model such as a recurrent neural network, a recursive neural network, or a convolutional network.

[0070] In an embodiment of the present application, training a target model based on a target image can be performed by obtaining data of a reachable space of annotated target images based on the target images, and training a target model based on the data of the reachable space of annotated target images to obtain a high-precision target model that can detect the reachable space of the image.

[0071] The technical solution of the present application is to obtain an input sample image set, wherein the sample image set includes a plurality of target images obtained by a reachable space data enhancement method based on ground processing provided by an embodiment of the present application; and train a target model based on the target images. Therefore, by adopting the above method of the present application, the target model is trained using the obtained logical target images, thereby ensuring the accuracy of the model after training. At the same time, a large number of target images are obtained by a reachable space data enhancement method based on ground processing provided by an embodiment of the present application, thereby reducing the cost of training the model.

[0072] See also Fig.13, which shows a reachable space data enhancement device based on ground processing provided by an embodiment of the present application, the device 300 includes: a first data set acquisition unit 310, a second data set acquisition unit 320, a target image acquisition unit 330 and a data enhancement unit 340. The first data set acquisition unit 310 is used to acquire a first data set, wherein the first data set includes a plurality of first images, each of which is respectively marked with a reachable space; the second data set acquisition unit 320 is used to acquire a second data set including a plurality of second images based on the number of the first images in the first data set, each of which is respectively marked with a reachable space, wherein the ground data corresponding to the reachable space marked in the second image is different from the ground data corresponding to the reachable space marked in the first image; the target image selection unit 330 is used to select a first target image from the first data set, and select a second target image from the second data set; the data enhancement unit 340 is used to process the ground data corresponding to the reachable space marked in the first target image based on the ground data corresponding to the reachable space marked in the second target image, so as to obtain a target image.

[0073] As an implementation mode, the first data set acquisition unit 310 obtains the first data set from the associated cloud via WiFi, wherein the first data set includes 1 million first images, each of which is marked with a reachable space by using a line to mark the intersection of the reachable space and the non-reachable space, and the ground data corresponding to the reachable space marked in each first image is cement ground data.

[0074] The second data set acquisition unit 320 acquires a second data set of multiple second images based on a preset selection ratio range of 1% to 3% set by the user and pre-stored in the second data set acquisition unit 320 and the number of first images in the first data set (1 million), and the number of the multiple second images is within a numerical range of 10,000 to 30,000 (1 million × 1% to 1 million × 3%) obtained by multiplying the preset selection ratio range and the number of the first images.

[0075] The target image selection unit 330 randomly selects a first target image from the first data set, and randomly selects a second target image from the second data set.

[0076] The data enhancement unit 340, if the first area corresponding to the reachable space marked in the first target image is larger than the second area corresponding to the reachable space marked in the second target image, obtain the overlap area between the first area and the second area; use the ground data of the overlap area in the second target image as the ground data of the first replacement area; select the ground data of the second replacement area from the ground data of the overlap area in the second target image, the size of the second replacement area is the same as the area in the first area except the overlap area; splice the ground data of the first replacement area with the ground data of the second replacement area to obtain the ground data of the target replacement area; use the ground data of the target replacement area to replace the ground data of the first area in the first target image, and obtain the target image. If the first area corresponding to the reachable space marked in the first target image is not larger than the second area corresponding to the reachable space marked in the second target image, obtain the overlap area between the first area and the second area; use the ground data of the overlap area in the second target image as the ground data of the target replacement area; use the ground data of the target replacement area to replace the ground data of the first area in the first target image, and obtain the target image.

[0077] The technical solution of the present application utilizes a small amount of special-shaped data to annotate a large amount of ordinary data, thereby obtaining a large number of target images annotated with reachable spaces and having different ground data corresponding to the reachable spaces, thereby enhancing the data type of the reachable spaces in a ground-based processing manner and reducing the cost of annotating the reachable spaces in the images.

[0078] See also Fig.14 , which shows a model training device provided by an embodiment of the present application, the device 400 includes: a sample image set acquisition unit 410 and a model training unit 420. The sample image set acquisition unit 410 is used to acquire an input sample image set, wherein the sample image set includes a plurality of target images obtained by a reachable space data enhancement method based on ground processing provided by an embodiment of the present application; the model training unit 420 is used to train a target model based on the target image.

[0079] The technical solution of the present application utilizes a large number of logical target images to train a target model, thereby improving the accuracy of the trained target model. At the same time, it utilizes a low-cost method to obtain the target image, thereby reducing the cost of training the target model.

[0080] It should be noted that each embodiment of this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. Any processing method described in the method embodiment can be implemented by the corresponding processing module in the device embodiment, and will not be repeated one by one in the device embodiment.

[0081] See also Fig.15 Based on the above-mentioned method for enhancing reachable space data based on ground processing, the present application also provides another electronic device 500 including a processor 510 that can execute the above-mentioned method for enhancing reachable space data based on ground processing, and the electronic device 500 also includes one or more processors 510, a memory 520, and one or more application programs. Among them, the memory 520 stores a program that can execute the content of the above-mentioned embodiment, and the processor 510 can execute the program stored in the memory 520. Among them, the electronic device 500 can be a vehicle, an intelligent control panel, a smart phone, a smart wearable device, an intelligent voice navigation device, an intelligent robot, a tablet computer, a personal computer, etc.

[0082] Among them, the processor 510 may include one or more cores for processing data and a message matrix unit. The processor 510 uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions and processes data of the electronic device 500 by running or executing instructions, programs, code sets or instruction sets stored in the memory 520, and calling data stored in the memory 520. Optionally, the processor 510 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 520 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor, but may be implemented separately through a communication chip.

[0083] The memory 520 may include a random access memory (RAM) or a read-only memory (ROM). The memory 520 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 520 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as obtaining a first data set, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a first target image, a second target image, a target image) created by the terminal during use, etc.

[0084] See also Fig.16 , which shows a structural block diagram of a computer-readable storage medium 600 provided in an embodiment of the present application. The computer-readable storage medium 600 stores a program code 610, and the program code 610 can be called by a processor to execute the method described in the above method embodiment.

[0085] The computer readable storage medium 600 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 600 includes a non-transitory computer-readable storage medium. The computer readable storage medium 600 has storage space for program codes 610 that perform any method steps in the above method. These program codes 610 can be read from or written to one or more computer program products. The program code can be compressed, for example, in an appropriate form.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for enhancing accessible space data based on ground processing, characterized in that: The method comprises: Acquire a first data set, wherein the first data set includes a plurality of first images, and each first image is respectively annotated with a reachable space; Based on the number of the first images in the first data set, obtaining a second data set including a plurality of second images, each of the second images being respectively annotated with a reachable space, wherein ground data corresponding to the reachable space annotated in the second images is different from ground data corresponding to the reachable space annotated in the first image; Selecting a first target image from the first data set, and selecting a second target image from the second data set; The ground data corresponding to the reachable space marked in the first target image is processed based on the ground data corresponding to the reachable space marked in the second target image to obtain a target image.

2. The method for enhancing accessible space data based on ground processing according to claim 1, characterized in that: The step of processing the ground data corresponding to the reachable space marked in the first target image based on the ground data corresponding to the reachable space marked in the second target image to obtain the target image includes: The ground data corresponding to the reachable space marked in the first target image is replaced with the ground data corresponding to the reachable space marked in the second target image to obtain a target image.

3. The method for enhancing accessible space data based on ground processing according to claim 2, characterized in that: The step of replacing the ground data corresponding to the reachable space marked in the first target image with the ground data corresponding to the reachable space marked in the second target image to obtain the target image includes: If a first area corresponding to the reachable space marked in the first target image is larger than a second area corresponding to the reachable space marked in the second target image, obtaining an overlapping area between the first area and the second area; Using the ground data of the overlapping area in the second target image as the ground data of the first replacement area; Selecting ground data of a second replacement area from ground data of the overlapped area in the second target image, wherein the size of the second replacement area is the same as that of an area in the first area except the overlapped area; splicing the ground data of the first replacement area with the ground data of the second replacement area to obtain ground data of a target replacement area; The ground data of the first area in the first target image is replaced by the ground data of the target replacement area to obtain a target image.

4. The method for enhancing accessible space data based on ground processing according to claim 3, characterized in that: If a first area corresponding to the reachable space marked in the first target image is not larger than a second area corresponding to the reachable space marked in the second target image, replacing ground data corresponding to the reachable space marked in the first target image with ground data corresponding to the reachable space marked in the second target image to obtain the target image includes: Acquire an overlapping area between the first area and the second area; Using the ground data of the overlapping area in the second target image as the ground data of the target replacement area; The ground data of the first area in the first target image is replaced by the ground data of the target replacement area to obtain a target image.

5. The method for enhancing accessible space data based on ground processing according to claim 1, characterized in that: The acquiring, based on the number of the first images in the first data set, a second data set including a plurality of second images comprises: A second data set including a plurality of second images is acquired based on a preset selection ratio range and the number of the first images in the first data set, wherein the number of the plurality of second images is within a numerical range obtained by multiplying the preset selection ratio range by the number of the first images.

6. A model training method, characterized in that: The method comprises: Obtaining an input sample image set, wherein the sample image set includes a plurality of target images obtained based on the method according to any one of claims 1 to 5; A target model is trained based on the target image.

7. A device for enhancing accessible space data based on ground processing, characterized in that: The device comprises: A first data set acquisition unit, configured to acquire a first data set, wherein the first data set includes a plurality of first images, each of which is respectively annotated with a reachable space; a second data set acquisition unit, configured to acquire, based on the number of the first images in the first data set, a second data set including a plurality of second images, each of the second images being respectively annotated with a reachable space, wherein ground data corresponding to the reachable space annotated in the second image is different from ground data corresponding to the reachable space annotated in the first image; a target image selection unit, configured to select a first target image from the first data set, and select a second target image from the second data set; A data enhancement unit is used to process the ground data corresponding to the reachable space marked in the first target image based on the ground data corresponding to the reachable space marked in the second target image to obtain a target image.

8. A model training device, characterized in that: The device comprises: A sample image set acquisition unit, used to acquire an input sample image set, wherein the sample image set includes a plurality of target images obtained based on the method according to any one of claims 1 to 5; A model training unit is used to train a target model based on the target image.

9. An electronic device, characterized in that: include: one or more processors; Memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1-6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 6.

Images