Image recognition optimization method and system based on machine learning

Abstract

The invention discloses an image recognition optimization method and system based on machine learning, and belongs to the technical field of image recognition, and the method comprises the steps: processing an original image to obtain a first image, and building a first training set based on the first image; training the migration identification model based on the first training set to obtain a first model; collecting a scene image, and performing image segmentation on the scene image based on the first model and the second model to generate a first planning direction and a second planning direction; if the first planning direction is similar to the second planning direction, the agricultural machine moves according to one planning direction, and an actual direction is generated in the moving process; if the actual direction deviates from the standard direction, correcting the moving direction of the agricultural machine, and uploading the scene image as a second image; and performing region segmentation on the second image by using the third model and generating a second training set, and continuing to train the first model based on the second training set. According to the invention, the operation efficiency of the agricultural machinery is improved, and the dependence on the network is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and in particular relates to an image recognition optimization method and system based on machine learning. Background Art

[0002] With the advancement of agricultural modernization, intelligent agricultural machinery has become an important means to improve agricultural production efficiency and reduce labor costs. As an important component of intelligent agricultural machinery, image recognition technology can accurately identify the environment in which agricultural machinery is located, and can automatically control agricultural machinery to perform reliable sowing or harvesting operations based on the recognition results.

[0003] Currently, cameras are generally installed on agricultural machinery, which collect scene images through the cameras and use deep learning algorithms to identify the scene images in order to determine the required operation area and plan the operation route. However, deep learning algorithms require a large number of images for pre-training. The available images for the specific field of agricultural machinery control are reduced and of poor quality, which makes the recognition effect of the trained deep learning algorithm poor. Manually collecting and labeling image data will take a lot of time and is inefficient. Summary of the invention

[0004] To solve the above problems, the present invention provides an image recognition optimization method and system based on machine learning to solve the problems in the prior art.

[0005] In order to achieve the above-mentioned object of the invention, the present invention proposes an image recognition optimization method based on machine learning, comprising: Acquire an original image, preprocess the original image to obtain a first image, and establish a first training set based on the first image; Selecting a migration identification model, and training the migration identification model based on the first training set to obtain a first model; The agricultural machine collects scene images of the current area at every first time period during the execution of the agricultural task, and after preprocessing the scene images, performs image segmentation on the scene images based on the first model and the second model in a local processor, and generates a first planning direction and a second planning direction for a future second time period according to the segmentation results; If the first planned direction is similar to the second planned direction, the agricultural machine moves according to one of the planned directions, obtains its own positioning information in real time during the movement, and generates an actual direction according to the positioning information; Setting a standard direction, correcting the moving direction of the agricultural machinery if the actual direction deviates from the standard direction, and uploading the scene image as a second image to a cloud server; After collecting a predetermined number of the second images, the cloud server uses the third model to perform region segmentation on the second images and generate a second training set, and continues to train the first model based on the second training set to continuously optimize the first model.

[0006] Furthermore, performing region segmentation on the second image comprises the following steps: The second image is converted into a grayscale image, the third model divides the second image into a plurality of initial regions, one of the initial regions is defined as a first region, and the other initial regions are defined as second regions, a first similarity between the first region and each of the second regions is calculated, the first similarity greater than a first threshold is screened, and the corresponding second region is defined as a third region, the third regions adjacent to each other are connected to generate a fourth region, and the first region is connected to the fourth region adjacent to itself to generate an extended region; Extract the first area from the initial area again, and generate the extended area of ​​the first area, repeat this step until all the initial areas are traversed, calculate the second similarities between the generated multiple extended areas, classify the extended areas into multiple area sets, the second similarities between the extended areas in the area sets are all greater than a second threshold, merge the first areas corresponding to the extended areas in the area sets into independent areas, and divide them into crop areas and non-crop areas based on the average value of the grayscale values ​​of the pixels contained in the independent areas.

[0007] Furthermore, segmenting the second image into the initial regions comprises the following steps: The grayscale value change of each pixel in the grayscale image in each direction is calculated by traversing, and if the change of the pixel is greater than a third threshold, the pixel is set as a boundary point, and the adjacent boundary points are connected in sequence to obtain a region segmentation line, and the second image is segmented into the initial region based on the region segmentation line.

[0008] Further, calculating the first similarity between the first region and the second region includes the following steps: Setting multiple value intervals, dividing the pixels in the first area into multiple first groups and the pixels in the second area into multiple second groups based on the value intervals and grayscale values, counting a first number of pixels in each of the first groups and a second number of pixels in each of the second groups, normalizing the first number and the second number, respectively, to obtain a first sequence and a second sequence, wherein the first sequence and the second sequence respectively include multiple first values ​​and multiple second values; Randomly selecting the same number of target points with the same grayscale value in the first area and the second area, generating a restricted range with each of the target points as the center, defining the pixel points other than the target point in the restricted range as auxiliary points, and calculating the distribution value of the target point based on the grayscale values ​​of the target point and the auxiliary points; A third sequence and a fourth sequence are generated based on the distribution values ​​of the first area and the second area respectively containing the target point, and the first similarity is calculated based on the third sequence and the fourth sequence.

[0009] Furthermore, the difference between the distribution values ​​of the target points corresponding to the third sequence and the fourth sequence is calculated, the summed average of the differences is calculated, the summed average is mapped to a value between 0 and 1, and the mapped value is used as the first similarity.

[0010] Furthermore, the second similarity between the two extended areas is calculated based on the number of overlapping pixels included in the two extended areas.

[0011] Further, generating the first planning direction or the second planning direction includes the following steps: Obtain a first boundary line and a second boundary line representing the crop area in the scene image after segmentation, wherein the first boundary line and the second boundary line are adjacent and perpendicular to each other, obtain the current forward direction of the agricultural machinery, and define an extension direction of the second boundary line with a smaller angle with the forward direction as the first planning direction or the second planning direction.

[0012] Furthermore, the second model recognizes the scene image including the following steps: The second model searches for a first direction, extracts multiple rows of pixel points along the first direction, plots the grayscale value changes of each row of pixel points in a coordinate system, obtains a fluctuation curve corresponding to each row of pixel points, locates coordinate points in each of the fluctuation curves that are adjacent in position and whose grayscale value difference is greater than a fourth threshold, selects a pixel point corresponding to one of the coordinate points and defines it as an anchor point, in the scene image, connects the anchor points in sequence along a second direction perpendicular to the first direction to obtain a first segmentation line, connects the endpoints of the first segmentation line in sequence along the first direction to obtain a second segmentation line, and divides the scene image into a crop area and a non-crop area based on the average value of the grayscale values ​​of the pixels in the area enclosed by the first segmentation line and the second segmentation line.

[0013] Furthermore, the migration identification model is a CNN or FCN model.

[0014] The present invention also provides an image recognition optimization system based on machine learning, which is used to implement the above-mentioned image recognition optimization method based on machine learning, and the system includes: A pre-training module, used to obtain an original image, pre-process the original image to obtain a first image, establish a first training set based on the first image, select a migration recognition model, and train the migration recognition model based on the first training set to obtain a first model; A processing module, wherein the agricultural machine collects scene images of the current area at every first time period during the execution of the agricultural task, and after preprocessing the scene images, a local processor in the processing module performs image segmentation on the scene images based on the first model and the second model, and generates a first planning direction and a second planning direction for a future second time period according to the segmentation results; Verification module, if the first planned direction is similar to the second planned direction, the agricultural machine moves according to one of the planned directions, obtains its own positioning information in real time during the movement, the verification module generates an actual direction according to the positioning information, sets a standard direction, corrects the moving direction of the agricultural machine if the actual direction deviates from the standard direction, and uploads the scene image as the second image to the cloud server; An optimization module, after collecting a predetermined number of the second images, the cloud server in the optimization module uses a third model to perform region segmentation on the second images and generate a second training set, and continues to train the first model based on the second training set to continuously optimize the first model.

[0015] Beneficial effects: The present invention provides accurate learning samples for the model by manually drawing contour segmentation lines and establishing a first training set, which helps to improve recognition accuracy. By introducing a training set in a specific field for retraining based on the migration model, the training cost is reduced and the convergence speed of the model is accelerated. The agricultural machinery collects images in real time when performing tasks, and uses the first model and the second model to perform image segmentation, generate two planning directions and verify each other, which increases the accuracy and robustness of direction planning.

[0016] In specific applications, the edge processor of agricultural machinery is used to generate planning directions, which reduces dependence on cloud servers, improves data processing speed, and can generate planning directions offline even in poor network conditions. By setting standard directions and combining positioning information, the direction of agricultural machinery is corrected. During the operation, scene images with poor recognition effects are uploaded to the cloud server, and a third model is used for automated region segmentation to generate a second training set, thereby continuously optimizing the first model. In summary, the present invention improves the operating efficiency and accuracy of agricultural machinery by combining machine learning and edge computing technologies, while reducing dependence on the network, enhancing the robustness and sustainability of the system, and optimizing the model without manual intervention after the model is deployed, thereby improving the level of automation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart of the steps of the image recognition optimization method based on machine learning of the present invention; Figure 2 It is a schematic diagram of merging initial regions of the present invention; Figure 3 Generating a schematic diagram of a first planning direction for the present invention; Figure 4 It is a schematic diagram of the change curve of the present invention; Figure 5 This is a structural diagram of the image recognition optimization system based on machine learning of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0019] like Figure 1 As shown, an image recognition optimization method based on machine learning includes: S1: Acquire an original image, preprocess the original image to obtain a first image, and establish a first training set based on the first image.

[0020] The original image is a farmland image involving various crops, such as wheat, rice, corn, etc., which is collected manually in advance. After the original image is collected, it is preprocessed. The preprocessing includes rotation, denoising and standardized cropping. The preprocessing makes the image more suitable for training deep learning models. The preprocessed original image is defined as the first image, and the contour segmentation line is manually drawn in the first image. All the first images are uniformly stored and defined as the first training set.

[0021] S2: Select a migration recognition model, and train the migration recognition model based on the first training set to obtain a first model.

[0022] In this embodiment, the migration recognition model is a CNN or FCN model.

[0023] The transfer recognition model is an image recognition model that has already been trained, such as CNN and FCN models. However, the training images used by the transfer recognition model are non-agricultural images. For example, urban road images are used for training. By introducing the first training set to retrain the already trained transfer recognition model, the subsequent training cost can be reduced and the convergence speed of the model during the training process can be accelerated.

[0024] S3: When performing agricultural tasks, the agricultural machinery collects scene images of the current area at intervals of a first time period. After preprocessing the scene images, the scene images are segmented based on the first model and the second model in the local processor, and the first planning direction and the second planning direction for the future second time period are generated according to the segmentation results.

[0025] Agricultural machinery includes harvesters, seeders, etc. Agricultural tasks include sowing, harvesting, weeding, etc. The following takes the harvesting task of the harvester as an example for explanation; first, an edge processor is set in the agricultural machinery, and when the agricultural machinery performs the harvesting task, the trained first model is used to obtain the scene image containing the farmland in front of the agricultural machinery using the camera set on the front of the agricultural machinery, and the scene image is sent to the edge processor. The edge processor collects the scene image every 3 seconds, and after preprocessing the scene image, the first model and the second model are used to segment the scene image, respectively, to obtain the first segmentation result and the second segmentation result, wherein the second model is segmented according to the gray value in the scene image. Finally, the first planning direction within the next 4 seconds is generated according to the first segmentation result, and the second planning direction is generated according to the second segmentation result.

[0026] Since the network status in the area where the farmland is located may be poor, the planning direction can be generated in the edge processor so that the data does not need to be uploaded to the cloud server, which improves the speed of obtaining image recognition results. When the network status is poor, the planning direction of the agricultural machinery can also be generated offline.

[0027] S4: If the first planned direction is similar to the second planned direction, the agricultural machine moves according to one of the planned directions, obtains its own positioning information in real time during the movement, and generates an actual direction according to the positioning information.

[0028] The following provides a method for determining whether the first planning direction and the second planning direction are similar. For example, the agricultural machinery needs to move in a direction parallel to the boundary of the farmland. Ideally, the generated first planning direction and the second planning direction should be parallel to the boundary of the farmland, and the two planning directions should completely overlap. Based on this goal, multiple reference points can be selected in the first planning direction to obtain the minimum distance between each reference point and the second planning direction, which is defined as the reference distance. The summed average of the reference distances of all reference points is used as the difference value between the two directions. The smaller the difference value, the better the overlap between the two directions, and the more similar they are. A judgment threshold can be set. When the difference value is less than the judgment threshold, the first planning direction and the second planning direction are judged to be similar. In addition, if the first planning direction and the second planning direction are not similar, the agricultural machinery stops moving and generates an alarm signal.

[0029] If the first planned direction is similar to the second planned direction, the agricultural machine selects the first planned direction to move. In other embodiments, the agricultural machine may also select the second planned direction to move. A GPS positioning module is provided in the agricultural machine, which can obtain its own position information, i.e., positioning information, in real time. During the movement of the agricultural machine, the actual direction is generated according to the positioning information.

[0030] S5: Setting a standard direction. If the actual direction deviates from the standard direction, correcting the moving direction of the agricultural machinery, and uploading the scene image as a second image to the cloud server.

[0031] The standard direction is a preset ideal harvesting direction. For example, the harvester needs to perform rotary harvesting parallel to the farmland boundary during the harvesting process. A standard farmland map is set in the agricultural machine. The agricultural machine generates an actual direction in the standard farmland map based on its own positioning information. When there is a large angle between the generated actual direction and the farmland boundary, the system determines that the actual direction deviates from the standard direction. At this time, the agricultural machine automatically corrects it based on its own positioning information and the boundary of the standard farmland map. In particular, this step is only used to automatically correct the moving direction of the agricultural machine, and is not directly used for direction planning. The reason is that using positioning information for direction planning will result in positioning errors, and there is also a situation where positioning cannot be performed due to poor network signals. The reliability of using image recognition for direction planning is higher.

[0032] S6: After collecting a predetermined number of second images, the cloud server uses the third model to perform region segmentation on the second images and generate a second training set, and continues to train the first model based on the second training set to continuously optimize the first model.

[0033] After determining that the planned direction of the agricultural machinery is wrong, the scene image of the planned direction is generated and uploaded to the cloud server as the second image. After collecting a certain number of second images, the motion server uses the third model to perform regional segmentation on the second image to generate a second training set. The images in the first training set are segmented using manual annotation to ensure the initial training effect of the first model. In the subsequent model optimization process, the third model performs image segmentation and annotation on the second image. In this way, the first model can be trained uninterruptedly during the operation of the agricultural machinery, thereby improving the optimization efficiency of the model. Finally, the cloud server will delegate the optimized first model to the edge processor each time, thereby continuously improving the image recognition effect.

[0034] The present invention provides accurate learning samples for the model by manually drawing contour segmentation lines and establishing a first training set, which helps to improve recognition accuracy. By introducing a training set in a specific field for retraining based on the migration model, the training cost is reduced and the convergence speed of the model is accelerated. The agricultural machinery collects images in real time when performing tasks, and uses the first model and the second model to perform image segmentation, generate two planning directions and verify each other, which increases the accuracy and robustness of direction planning.

[0035] In specific applications, the edge processor of agricultural machinery is used to generate planning directions, which reduces dependence on cloud servers, improves data processing speed, and can generate planning directions offline even in poor network conditions. By setting standard directions and combining positioning information, the direction of agricultural machinery is corrected. During the operation, scene images with poor recognition effects are uploaded to the cloud server, and a third model is used for automatic region segmentation to generate a second training set, thereby continuously optimizing the first model. In summary, the present invention improves the operating efficiency and accuracy of agricultural machinery by combining machine learning and edge computing technologies, while reducing dependence on the network, enhancing the robustness and sustainability of the system, and after the model is deployed, the model can be optimized without human intervention, thereby improving the level of automation.

[0036] In this embodiment, performing region segmentation on the second image includes the following steps: The second image is converted into a grayscale image. The third model divides the second image into multiple initial regions, defines one of the initial regions as the first region, and the remaining initial regions as the second regions. The first similarity between the first region and each second region is calculated, and the first similarity greater than the first threshold is screened. The corresponding second region is defined as the third region, and the adjacent third regions are connected to generate the fourth region. The first region is connected to the fourth region adjacent to itself to generate an extended region.

[0037] Combination Figure 2 The steps are described in detail in Figure 2In , the second image is segmented into multiple initial regions, and the specific segmentation method is described later. Each initial region has a number. Due to the influence of factors such as obstacles, weeds, and lighting, the same region may be segmented into multiple regions; to solve this problem, first select the initial region numbered 1, defined as the first region, and the remaining initial regions are the second regions, and calculate the first similarity between the first region and the second region. The first similarity is the similarity of grayscale distribution. The greater the first similarity between the first region and the second region, the greater the possibility that the same object is contained therein, for example, both include farmland, or both are soil gaps between farmland. By comparison, it is determined that the first similarity of the first region is greater than the first threshold region numbered 2, 3, 5, 6, 8, so the second region 2, 3, 5, 6, 8 is defined as the third region. Among them, the third regions numbered 2 and 3 are adjacent, and the third regions numbered 5 and 6 are adjacent. The third regions numbered 2 and 3 are merged into the fourth region 1, and the third regions numbered 5 and 6 are merged into the fourth region 2. The fourth region 1 is adjacent to the first region, and the first region and the fourth region 1 are merged into the extended region 1.

[0038] The first area is extracted from the initial area again, and an extended area of ​​the first area is generated. This step is repeated until all initial areas are traversed, and the second similarities between the generated multiple extended areas are calculated. The extended areas are classified into multiple area sets. The second similarities between the extended areas in the area sets are all greater than the second threshold. The first areas corresponding to the extended areas in the area sets are merged into independent areas, and the independent areas are divided into crop areas and non-crop areas based on the average grayscale values ​​of the pixels contained in the independent areas.

[0039] Then, the initial region numbered 2 is extracted as the first region, and the extended region of the first region is generated. It is assumed that the extended region 2 corresponding to the initial region numbered 2 includes numbers 1, 2, and 3, and the extended region 3 corresponding to the initial region numbered 3 includes numbers 1, 2, 3, and 9. After that, the second similarity between each extended region is calculated. After screening, the second similarity between extended regions 1, 2, and 3 is greater than the second threshold. Extended regions 1, 2, and 3 correspond to initial regions 1, 2, and 3, respectively, that is, first regions 1, 2, and 3, so they are merged into independent regions. Finally, according to the average grayscale value of the pixels in the independent regions, they are divided into crop regions and non-crop regions.

[0040] In this embodiment, segmenting the second image into initial regions includes the following steps: The grayscale value change of each pixel in the grayscale image in each direction is calculated by traversing. If the change of the pixel is greater than the third threshold, the pixel is set as a boundary point, and the adjacent boundary points are connected in sequence to obtain a region segmentation line, and the second image is segmented into initial regions based on the region segmentation line.

[0041] Specifically, the difference between the grayscale values ​​of each pixel and its adjacent pixels is calculated. The difference is the change. If the difference is greater than the third threshold, it indicates that the grayscale value at the pixel has increased or decreased significantly. The pixel may represent the boundary between the crop and the soil. After that, the adjacent boundary points are connected in sequence to obtain the following: Figure 2 The partitioning results are shown.

[0042] In this embodiment, calculating the first similarity between the first region and the second region includes the following steps: A plurality of numerical intervals are set, and the pixels in the first area are divided into a plurality of first groups and the pixels in the second area are divided into a plurality of second groups based on the numerical intervals and the grayscale values, and a first number of pixels contained in each first group and a second number of pixels contained in each second group are counted, and the first number and the second number are normalized respectively to obtain a first sequence and a second sequence, wherein the first sequence and the second sequence respectively include a plurality of first numerical values ​​and a plurality of second numerical values.

[0043] The size of the numerical interval can be determined based on actual experience. Since the grayscale value is between 0 and 255, it can be divided into 5 numerical regions, and each numerical interval contains 51 numerical values. Then, the grayscale values ​​of each pixel in the first region and the second region are obtained, and the pixel is classified into the corresponding numerical interval according to the grayscale value, so that the pixel is divided into multiple first groups and second groups. For example, the first group 1 includes 20 pixels, and the corresponding first numerical value is 20, and the second group 1 includes 25 pixels, and the corresponding second numerical value is 25. In order to eliminate the difference in order of magnitude, after the pixels are grouped, they are also normalized respectively, and the numerical values ​​are normalized to between 0 and 1. In this embodiment, the maximum and minimum normalization processes are used for the five first quantities and the five second quantities respectively, and the first sequence corresponding to the five first quantities and the second sequence corresponding to the five second quantities are obtained, and the numerical values ​​included therein are defined as the first numerical value or the second numerical value respectively.

[0044] The same number of target points with the same grayscale values ​​are randomly selected in the first area and the second area, and a restriction range is generated with each target point as the center. The pixel points except the target point within the restriction range are defined as auxiliary points, and the distribution value of the target point is calculated based on the grayscale values ​​of the target point and the auxiliary point.

[0045] A third sequence and a fourth sequence are generated based on the distribution values ​​of the target points contained in the first region and the second region, respectively, and a first similarity is calculated based on the third sequence and the fourth sequence.

[0046] The difference between the target point distribution values ​​corresponding to the third sequence and the fourth sequence is calculated, the average value of the difference is calculated, the average value is mapped to between 0 and 1, and the mapped value is used as the first similarity.

[0047] Select 5 target points in the first area and the second area respectively. When selecting, select the ones with the same grayscale value. For example, the grayscale value of target point 1 in the first area is the same as that of target point 1 in the second area, and the grayscale value of target point 2 in the first area is the same as that of target point 2 in the second area. Then, generate a 5*5 restriction range with each target point as the center, and include 25 pixels in the restriction range. The pixels other than the target points are auxiliary points. Then, calculate the distribution law of the pixels around each target point according to the grayscale values ​​of the target point and the auxiliary points.

[0048] Specifically, this embodiment calculates the distribution value Y of the target point in the first area and the second area based on the following first formula. The first formula is as follows: , if the target point is in the first area, P is the first value corresponding to the first group where the target point is located; if it is in the second area, P is the second value corresponding to the second group where the target point is located; N is the number of auxiliary points within the limit range; R0 is the gray value of the target point; Rn is the gray value of the nth auxiliary point; Dn is the distance between the target point and the nth pixel point; ω1 and ω2 are the first weight and the second weight, respectively. In the first formula, if the distribution value of the target point in the first area is to be calculated, the gray value of the target point is first obtained, and the first group in which the target point is located is determined according to the gray value, and then the first value corresponding to the group is obtained as the value of P. If the corresponding distribution values ​​of the two target points in the first area and the second area are closer, it indicates that the distribution law of the pixels in the corresponding limit range is closer.

[0049] Based on this, when comparing the third sequence and the fourth sequence, the difference in the distribution values ​​between the target points is first calculated, and then the summed average of the differences is calculated, and then the summed average is converted into the first similarity according to the third formula. The third formula is y=1 / (1+|x|), where y is the converted value and x is the value before conversion. When x is equal to 0, the first similarity reaches the maximum, which is 1.

[0050] In this embodiment, the second similarity between the two extended areas is calculated based on the number of overlapping pixels between the two extended areas: Specifically, the second similarity between the extended areas is calculated based on the following second formula, where the two extended areas to be compared are defined as the first extended area and the second extended area, respectively. The second formula is: , a is the number of overlapping pixels in the first extension area and the second extension area, A is the number of pixels in the first extension area, and B is the number of pixels in the second extension area. For example, if the number of pixels in the first extension area is 120, the number of pixels in the second extension area is 150, and the number of overlapping pixels between the first extension area and the second extension area is 100, then the corresponding second similarity is 0.75.

[0051] In this embodiment, generating the first planning direction or the second planning direction includes the following steps: The first boundary line and the second boundary line representing the crop area in the segmented scene image are obtained, the first boundary line and the second boundary line are adjacent and perpendicular to each other, the current forward direction of the agricultural machinery is obtained, and the extension direction of the second boundary line with a smaller angle with the forward direction is defined as the first planning direction or the second planning direction.

[0052] The current forward direction of the agricultural machine can be determined based on the direction when it stopped after the last movement, and then the first boundary line and the second boundary line of the crop area are obtained, such as Figure 3 As shown, it includes the first boundary line C1 and the second boundary line C2, and the current forward direction of the agricultural machine is C3. Since the angle between the forward direction C3 and the second boundary line C2 is small, the first planned direction or the second planned direction is adjusted to be parallel to the second boundary line C2. In addition, when the agricultural machine reaches the boundary point of the crop coverage area, it turns 90 degrees and continues to produce in the planned direction. Other restriction adjustments can be added according to actual conditions and will not be introduced here.

[0053] In this embodiment, the second model recognizes the scene image including the following steps: The second model searches for the first direction, extracts multiple rows of pixels along the first direction, plots the changes in the grayscale values ​​of each row of pixels in the coordinate system, obtains a fluctuation curve corresponding to each row of pixels, locates the coordinate points in each fluctuation curve that are adjacent and whose grayscale value difference is greater than the fourth threshold, selects the pixel point corresponding to one of the coordinate points as an anchor point, connects the anchor points in sequence along a second direction perpendicular to the first direction in the scene image to obtain a first segmentation line, connects the endpoints of the first segmentation line in sequence along the first direction to obtain a second segmentation line, and divides the scene image into crop areas and non-crop areas based on the average grayscale values ​​of the pixels in the area enclosed by the first segmentation line and the second segmentation line.

[0054] Since the shape of farmland generally leaves gaps between crops, after finding the first direction perpendicular to the extension direction of the crop gaps, the grayscale value of the pixel point will appear as follows: Figure 4 When obtaining the first direction, each direction in the scene image can be traversed to draw a change curve. When the generated change curve meets the Figure 4When the form is in, it indicates that the correct first direction has been found. Locate the coordinate points whose adjacent ordinate axis differences in each fluctuation curve are greater than the fourth threshold, and select the pixel point corresponding to one of the coordinate points as the anchor point. The purpose is to find the boundary point between the crop and the soil, for example Figure 3 The coordinate point E1 and the coordinate point E2 in the image are connected, and finally the pixel points corresponding to the coordinate point E1 and the coordinate point E2 are connected along the second direction, the second direction is perpendicular to the first direction, and finally the first dividing line between the crops and the soil in the farmland is obtained. The endpoints of the first dividing line are connected along the first direction to obtain the dividing line between the farmland areas.

[0055] like Figure 5 As shown, the present invention also provides an image recognition optimization system based on machine learning, which is used to implement the above-mentioned image recognition optimization method based on machine learning, and the system includes: A pre-training module is used to obtain an original image, pre-process the original image to obtain a first image, establish a first training set based on the first image, select a migration recognition model, train the migration recognition model based on the first training set, and obtain a first model; The processing module collects scene images of the current area at first time intervals during the agricultural task performed by the agricultural machine. After preprocessing the scene images, the local processor in the processing module performs image segmentation on the scene images based on the first model and the second model, and generates the first planning direction and the second planning direction for the future second time interval according to the segmentation results. Verification module: if the first planned direction is similar to the second planned direction, the agricultural machine moves according to one of the planned directions, and obtains its own positioning information in real time during the movement. The verification module generates an actual direction based on the positioning information and sets a standard direction. If the actual direction deviates from the standard direction, the movement direction of the agricultural machine is corrected, and the scene image is uploaded to the cloud server as the second image; The optimization module, after collecting a predetermined number of second images, the cloud server in the optimization module uses the third model to perform region segmentation on the second images and generate a second training set, and continues to train the first model based on the second training set to continuously optimize the first model.

[0056] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image recognition optimization method based on machine learning, characterized in that: include: Acquire an original image, preprocess the original image to obtain a first image, and establish a first training set based on the first image; Selecting a migration identification model, and training the migration identification model based on the first training set to obtain a first model; The agricultural machine collects scene images of the current area at every first time period during the execution of the agricultural task, and after preprocessing the scene images, performs image segmentation on the scene images based on the first model and the second model in a local processor, and generates a first planning direction and a second planning direction for a future second time period according to the segmentation results; If the first planned direction is similar to the second planned direction, the agricultural machine moves according to one of the planned directions, obtains its own positioning information in real time during the movement, and generates an actual direction according to the positioning information; Setting a standard direction, correcting the moving direction of the agricultural machinery if the actual direction deviates from the standard direction, and uploading the scene image as a second image to a cloud server; After collecting a predetermined number of the second images, the cloud server uses the third model to perform region segmentation on the second images and generate a second training set, and continues to train the first model based on the second training set to continuously optimize the first model.

2. The method according to claim 1, characterized in that Performing region segmentation on the second image comprises the following steps: The second image is converted into a grayscale image, the third model divides the second image into a plurality of initial regions, one of the initial regions is defined as a first region, and the other initial regions are defined as second regions, a first similarity between the first region and each of the second regions is calculated, the first similarity greater than a first threshold is screened, and the corresponding second region is defined as a third region, the third regions adjacent to each other are connected to generate a fourth region, and the first region is connected to the fourth region adjacent to itself to generate an extended region; Extract the first area from the initial area again, and generate the extended area of ​​the first area, repeat this step until all the initial areas are traversed, calculate the second similarities between the generated multiple extended areas, classify the extended areas into multiple area sets, the second similarities between the extended areas in the area sets are all greater than a second threshold, merge the first areas corresponding to the extended areas in the area sets into independent areas, and divide them into crop areas and non-crop areas based on the average value of the grayscale values ​​of the pixels contained in the independent areas.

3. The method according to claim 2, characterized in that Segmenting the second image into the initial regions comprises the following steps: The grayscale value change of each pixel in the grayscale image in each direction is calculated by traversing, and if the change of the pixel is greater than a third threshold, the pixel is set as a boundary point, and the adjacent boundary points are connected in sequence to obtain a region segmentation line, and the second image is segmented into the initial region based on the region segmentation line.

4. The method according to claim 2, characterized in that: Calculating the first similarity between the first region and the second region comprises the following steps: Setting multiple value intervals, dividing the pixels in the first area into multiple first groups and the pixels in the second area into multiple second groups based on the value intervals and grayscale values, counting a first number of pixels in each of the first groups and a second number of pixels in each of the second groups, normalizing the first number and the second number, respectively, to obtain a first sequence and a second sequence, wherein the first sequence and the second sequence respectively include multiple first values ​​and multiple second values; Randomly selecting the same number of target points with the same grayscale value in the first area and the second area, generating a restricted range with each of the target points as the center, defining the pixel points other than the target point in the restricted range as auxiliary points, and calculating the distribution value of the target point based on the grayscale values ​​of the target point and the auxiliary points; A third sequence and a fourth sequence are generated based on the distribution values ​​of the first area and the second area respectively containing the target point, and the first similarity is calculated based on the third sequence and the fourth sequence.

5. The method according to claim 4, characterized in that The difference between the distribution values ​​of the target point corresponding to the third sequence and the fourth sequence is calculated, the summed average of the differences is calculated, the summed average is mapped to a value between 0 and 1, and the mapped value is used as the first similarity.

6. The method according to claim 2, characterized in that: The second similarity between the two extended areas is calculated based on the number of overlapping pixels included in the two extended areas.

7. The method according to claim 2, characterized in that: Generating the first planning direction or the second planning direction comprises the following steps: Obtain a first boundary line and a second boundary line representing the crop area in the scene image after segmentation, wherein the first boundary line and the second boundary line are adjacent and perpendicular to each other, obtain the current forward direction of the agricultural machinery, and define an extension direction of the second boundary line with a smaller angle with the forward direction as the first planning direction or the second planning direction.

8. The method according to claim 7, characterized in that The second model recognizes the scene image including the following steps: The second model searches for a first direction, extracts multiple rows of pixel points along the first direction, plots the grayscale value changes of each row of pixel points in a coordinate system, obtains a fluctuation curve corresponding to each row of pixel points, locates coordinate points in each of the fluctuation curves that are adjacent in position and whose grayscale value difference is greater than a fourth threshold, selects a pixel point corresponding to one of the coordinate points and defines it as an anchor point, in the scene image, connects the anchor points in sequence along a second direction perpendicular to the first direction to obtain a first segmentation line, connects the endpoints of the first segmentation line in sequence along the first direction to obtain a second segmentation line, and divides the scene image into a crop area and a non-crop area based on the average value of the grayscale values ​​of the pixels in the area enclosed by the first segmentation line and the second segmentation line.

9. The method according to claim 1, characterized in that: The migration recognition model is a CNN or FCN model.

10. An image recognition optimization system based on machine learning, used to implement the method according to any one of claims 1 to 8, characterized in that: include: A pre-training module, used to obtain an original image, pre-process the original image to obtain a first image, establish a first training set based on the first image, select a migration recognition model, and train the migration recognition model based on the first training set to obtain a first model; A processing module, wherein the agricultural machine collects scene images of the current area at every first time period during the execution of the agricultural task, and after preprocessing the scene images, a local processor in the processing module performs image segmentation on the scene images based on the first model and the second model, and generates a first planning direction and a second planning direction for a future second time period according to the segmentation results; Verification module, if the first planned direction is similar to the second planned direction, the agricultural machine moves according to one of the planned directions, obtains its own positioning information in real time during the movement, the verification module generates an actual direction according to the positioning information, sets a standard direction, corrects the moving direction of the agricultural machine if the actual direction deviates from the standard direction, and uploads the scene image as the second image to the cloud server; An optimization module, after collecting a predetermined number of the second images, the cloud server in the optimization module uses a third model to perform region segmentation on the second images and generate a second training set, and continues to train the first model based on the second training set to continuously optimize the first model.

Images